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EXECUTIVE  SUMMARY 


This  technical  review  of  the  Capability  Portfolio  Analysis  Tool  (CP AT)  was  conducted 
by  the  Naval  Postgraduate  School  (NPS)  at  the  request  of  the  Program  Executive  Office 
(PEO)  Ground  Combat  Systems  (GCS).  The  CPAT  is  intended  to  support  Department  of 
the  Army  portfolio  resourcing  and  capabilities  decisions  for  the  ground  combat  fleet.  The 
tool  is  designed  to  identify  the  optimum  courses  of  action  (cost,  schedule,  and 
perfonnance)  for  portfolio  investment. 

The  development  of  CPAT  was  a  multiagency  collaboration  consisting  of  researchers  and 
analysts  from  PEO  GCS;  Anny  Materiel  Systems  Analysis  Activity  (AMSAA);  Sandia 
National  Labs;  Booz  Allen;  and  Tank  Automotive  Research,  Development  and 
Engineering  Center  (TARDEC).  Other  contributing  agencies  included  Maneuver  Center 
of  Excellence  (MCoE),  Fires  Center  of  Excellence  (FCoE),  Army  Medical  Department 
(AMEDD),  Combined  Arms  Support  Command  (CASCOM),  Training  and  Doctrine 
Command  Analysis  Center-White  Sands  Missile  Range  (TRAC-WSMR),  and  Project 
Managers  (PMs)  for  the  Abrams,  Bradley,  Field  Studies  Program  (FSP),  Ml  13,  Ground 
Combat  Vehicle  (GCV),  and  Stryker. 

This  validation  and  verification  report  provides  a  technical  review  of  CPAT’s 
performance  and  optimization  model.  At  its  core,  CPAT  is  an  optimization  model; 
however,  the  CPAT  objective  function  is  derived  using  multiattribute  decision- analysis 
techniques.  Therefore,  our  assessment  focuses  first  on  the  validity  of  the  value  (or 
perfonnance)  model  used  to  determine  the  objective  function  coefficients.  Secondly,  we 
verify  the  CPAT  optimization  fonnulation  and  the  Open  Programming  Language  (OPL) 
code  implementation.  Third,  we  validate  the  CPAT  model  by  varying  the  most  sensitive 
parameters  and  observe  the  changes  to  the  CPAT  solutions. 

The  assessment  of  the  value  model  includes  a  qualitative  examination  of  the  model’s 
structure  and  a  synthesis  of  the  quantitative  components  of  the  model.  Additionally,  the 
perfonnance  evaluation  assessment  includes  a  sensitivity  analysis  of  the  weights  and  a 
quantitative  analysis  of  the  value  function  range  usage. 

The  optimization  model  assessment  includes  a  thorough  examination  of  the  optimization 
formulation  and  implementation  tests  on  model  parameters  using  a  reduced  data  set  of 
three  roles:  the  Main  Battle  Tank  (MBT),  Infantry  Fighting  Vehicle  (IFV),  and  Cavalry 
(CAV)  vehicle  variants.  This  reduced  set  enables  an  examination  of  many  excursions  in  a 
limited  time  and  easier  implementation  of  the  results.  Conclusions  concerning  particular 
weapons  systems  should  be  avoided. 

Overall,  the  results  of  our  analysis  indicate  that  the  perfonnance  model  provides  robust 
results,  primarily  due  to  the  small  number  of  alternatives  considered  for  a  given  role. 
Moderate  changes  to  the  attribute  weight  or  the  shape  of  the  value  function  affect  a  small 
number  of  vehicle  alternatives  for  a  given  role.  Our  analysis  also  highlights  that  the 
model  contains  many  attributes  that  are  not  relevant  or  do  not  contribute  to  the 
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differentiation  among  alternatives.  For  example,  the  actual  range  of  the  data  for 
Maximum  Speed  in  the  CAV  role  utilizes  just  2%  of  the  possible  range  allowed  for  the 
value  function.  In  other  cases,  all  the  alternatives  receive  the  same  value  level  for  a 
qualitative  measure  or  only  one  or  two  alternatives  receive  any  value  from  a  binary 
measure.  Additionally,  attributes  should  be  weighted  globally  across  all  roles.  Currently, 
the  weighting  structure  results  in  20  different  models,  with  one  model  corresponding  to 
each  role.  Due  to  this  current  limitation,  alternative  values  from  one  model  should  not  be 
directly  compared  to  values  of  another  model. 

The  graphical  user  interface  (GUI)  on  the  optimization  model  does  allow  CP  AT  to  be 
used  by  novice  users  not  accustomed  to  running  optimization  models.  Therefore,  with 
little  training,  the  PMs  should  be  able  to  use  CP  AT  to  examine  fielding  and 
modernization  issues  as  long  as  custom  output  is  not  required. 

It  is  also  important  to  highlight  that  in  the  optimization  model  the  alternatives  compete 
across  roles;  unlike  in  the  value  model,  where  they  compete  amongst  each  other  within  a 
role.  Therefore,  the  value  trade-offs  are  critical  in  a  reduced  budget  environment.  The 
sensitivity  tests  of  the  optimization  model  also  indicate  that  CP  AT  results  are  highly 
sensitive  to  research,  development,  test  and  evaluation  (RDTE)  funding  profiles  for 
different  vehicle  alternatives  and  to  maximum  purchases  allowed  on  the  fielding 
schedule. 

Our  review  identifies  two  main  recommendations  that  can  be  addressed  within  the  current 
model  in  the  near  term.  First,  study  the  effects  of  removing  ineffective  value  measures. 
This  would  include  measures  that  do  not  use  the  full  range  of  the  scale  as  well  as 
qualitative  and  binary  measures  that  show  little  variation  in  scale.  Second,  aggregate 
CP  AT  at  the  brigade-set  level.  Currently,  all  fielding  and  modernization  decisions  are 
being  made  at  the  brigade-set  level,  implying  that  all  production  decisions  must  also  be 
made  at  the  brigade-set  level.  This  aggregation  would  allow  for  the  removal  of  the 
complicating  constraint  (1.27),  permitting  quicker  solution  times  and  smaller  optimality 
gaps. 

In  the  longer  term,  we  recommend  determining  a  “single”  objective  value  model  by 
reducing  the  number  of  attributes  and  reweighting  across  all  roles.  This  will  allow  users 
to  make  direct  value  comparisons  between  roles  and  strengthen  the  additive  assumptions 
implied  by  the  optimization.  Secondly,  develop  CP  AT  Version  2  to  consider  separate 
decisions  for  production  and  allow  inventory  to  be  carried  from  one  period  to  the  next. 
This  next-generation  CPAT  will  then  decouple  the  fielding  of  Brigade  (BDE)  sets  from 
the  vehicle  production  decisions  and  provide  for  lower  cost,  long-run  solutions  with 
greater  fleet  perfonnance. 
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1. 


BACKGROUND 


The  Capability  Portfolio  Analysis  Tool  (CP AT)  was  developed  following  a 
request  by  Program  Executive  Office  (PEO)  Ground  Combat  Systems  (GCS)  for  a  tool  to 
support  Department  of  the  Army  (DA)  portfolio  resourcing  and  capabilities  decisions  for 
the  ground  combat  fleet. 

Under  the  sponsorship  of  Mr.  Scott  Davis,  the  development  of  CPAT  was  a 
multiagency  collaboration  consisting  of  researchers  and  analysts  from  PEO  GCS,  Army 
Materiel  Systems  Analysis  Activity  (AMSAA),  Sandia  National  Labs,  Booz  Allen,  and 
Tank  Automotive  Research,  Development  and  Engineering  Center  (TARDEC).  Other 
contributing  agencies  included  Maneuver  Center  of  Excellence  (MCoE),  Fires  Center  of 
Excellence  (FCoE),  Army  Medial  Department  (AMEDD),  Combined  Arms  Support 
Command  (CASCOM),  Training  and  Doctrine  Command  Analysis  Center-White  Sands 
Missile  Range  (TRAC-WSMR),  and  Program  Managers  (PMs)  for  the  Abrams,  Bradley, 
Field  Studies  Program  (FSP),  Ml  13,  Ground  Combat  Vehicle  (GCV),  and  Stryker. 

The  CPAT  team  launched  Phase  I  of  development  in  Fiscal  Year  (FY)  2010. 
Booz  Allen  Hamilton  took  the  lead  role  on  the  performance  evaluation  component — with 
data  availability  input  from  AMSAA — while  Sandia  National  Labs  led  the  optimization 
analysis.  The  cost  analysis  was  provided  by  the  Tank-automotive  and  Armaments 
Command  (TACOM)  Cost  and  Systems  Analysis  Office.  Cost  data  was  provided  by  the 
PM  cost  teams  using  the  annual  weapons  systems  reviews.  The  schedule  was  provided  by 
the  individual  PM  offices. 

To  date,  CPAT  has  been  reviewed  and  accepted  the  following  senior  leadership 
within  the  analytical  community  (Edwards,  2011b). 


LTG  Walker 

Training  and  Doctrine  Command 

Dr.  Crain 

Army  Materiel  Systems  Analysis  Activity 

Dr.  Markowitz 

Army  headquarters  staff  (G3/5/7) 

MG  Spoehr 

G8-Program  Analysis  and  Evaluation 

BG  Dyess 

G8-Force  Development 

LTG  Phillips 

ASA-Acquisitions,  Logistics,  and  Technology 

Mr.  Bagby 

DASA-Cost  and  Economics 

Mr.  Ahem 

OSD-Acquisition,  Technology,  and  Logistics 

Dr.  Spruill 

Acquisition  Resources  and  Analysis 

1.1  VERIFICATION  AND  VALIDATION  (V&V)  METHODOLOGY 

We  recognize  for  our  analysis  that  CPAT  is  an  optimization  model  that  is 
characterized  by  mathematical  constraints  and  a  mathematical  objective  function.  To 
detennine  the  parameters  or  fixed  data  of  the  objective  function,  a  second  model  is 
analyzed  that  we  refer  to  as  the  CPAT  perfonnance  model.  The  perfonnance  model 
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provides  the  objective  function  coefficients  for  the  optimization  model’s  objective 
function. 

Our  intent  for  this  report  is  two-fold.  The  first  is  to  provide  a  verification  of 
CP  AT.  The  second  is  to  provide  model  stakeholders  with  the  means  to  validate  CP  AT. 
The  official  Army  definitions  of  V&V  follow  (Headquarters,  Department  of  the  Anny, 
2005,  p.  8): 

Verification  is  the  process  of  determining  if  the  M&S  accurately  represents  the 
developer’s  conceptual  description  and  specifications  and  meets  the  needs  stated 
in  the  requirements  document.  The  verification  process  evaluates  the  extent  to 
which  the  M&S  has  been  developed  using  sound  and  established  software 
engineering  techniques,  and  establishes  whether  the  M&S  logic  and  code 
correctly  perform  the  intended  functions. 

Validation  is  the  process  of  detennining  the  extent  to  which  the  M&S  adequately 
represents  the  real-world  from  the  perspective  of  its  intended  use. 

We  believe  the  Anny’s  V&V  definitions  are  inadequate  for  the  CP  AT  technical 
evaluation.  Therefore,  we  rely  on  the  definitions  from  the  National  Research  Council 
(2012,  p.  8)  to  frame  the  CP  AT  V&V: 

Verification:  The  process  of  detennining  how  accurately  a  computer  program 
(“code”)  correctly  solves  the  equations  of  the  mathematical  model.  This  includes 
code  verification  (determining  whether  the  code  conectly  implements  the 
intended  algorithms)  and  solution  verification  (detennining  the  accuracy  with 
which  the  algorithms  solve  the  mathematical  model’s  equations  for  specified 
quantities  of  interest). 

Validation:  The  process  of  determining  the  degree  to  which  a  model  is  an  accurate 
representation  of  the  real  world  from  the  perspective  of  the  intended  uses  of 
the  model. 

The  NRC  includes  a  third  category  not  commonly  recognized  by  the  military — 
the  Uncertainty  Quantification,  which  asks,  how  do  the  various  sources  of  enor  and 
uncertainty  feed  into  uncertainty  in  the  model-based  prediction  of  the  quantities  of 
interest?  Figure  1  shows  our  view  of  how  CP  AT  represents  the  true  system  through  the 
equations  of  the  optimization  fonnulation  and  the  Excel/Visual  Basic  for  Applications 
(VBA)  implementations  of  the  performance  model.  We  verify  that  the  computational 
models  are  correct,  i.e.,  they  are  implemented  according  to  the  qualitative  value  model 
and  the  optimization  equations. 
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Figure  1.  Verification,  validation,  and  prediction  as  they  relate  to  the  true,  physical 
system,  the  mathematical  model,  and  the  computational  model.  (Adapted  from 
AIAA  [1998]  and  the  NRC  [2012]). 

For  the  validation,  our  focus  is  on  the  technical  aspects  of  the  model.  We  exercise 
the  model  to  provide  results  that  are  useful  to  model  stakeholders  so  that  they  may  better 
understand  the  limitations  of  CP  AT.  Our  intent  for  validation  is  not  to  determine  if  the 
answers  provided  given  our  input  data  make  sense  from  a  policy  view.  We  leave  that 
analysis  to  many  others  who  have  the  subject  matter  expertise  and  data  to  make  those 
evaluations.  We  also  do  not  attempt  to  conduct  an  uncertainty  quantification  on  CPAT  for 
this  report. 

In  the  end,  our  CPAT  verification  should  answer  the  question,  “Flow  accurately 
does  the  computation  solve  the  underlying  equations  of  the  model  for  the  quantities  of 
interest?”  Our  hope  is  that  the  CPAT  validation  provides  enough  data  and  analysis  for 
model  stakeholders  to  answer  the  question,  “How  accurately  does  the  model  represent 
reality  for  the  quantities  of  interest?” 

1.2  CAPABILITY  PORTFOLIO  ANALYSIS  TOOL  (CPAT)  OVERVIEW 

CPAT  is  an  acquisition  tool  designed  to  identify  the  optimum  courses  of  action 
(cost,  schedule  and  performance)  for  portfolio  investment.  CPAT  provides  transparent 
and  replicable  analysis,  evaluating  courses  of  action  within  a  given  budget.  It  was 
designed  to  complement  the  formal  Analysis  of  Alternative  (AoA)  process  and  support 
the  transformation  and  modernization  of  the  combat  vehicle  fleet  (Edwards,  2011a). 
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The  CP  AT  performance  model  detennines  a  value  for  47  vehicles  across  20  roles 
using  49  attributes.1  However,  not  all  vehicles  are  considered  for  all  roles  and,  in  most 
cases,  no  more  than  four  to  six  vehicles  are  considered  for  a  given  role.  For  example,  the 
Turretless  GCV  in  the  CAV  role  will  not  be  considered  for  the  Infantry  Fighting  Vehicle 
(IFV)  role.  Table  1  lists  the  Heavy  Brigade  Combat  Team  and  Stryker  Brigade  Combat 
Team  (HBTC/SBCT)  vehicle  alternatives  by  role. 

Table  1.  Vehicle  alternatives. 


|  HBCT  Roles  I 

Main 

Battle 

Tank 

Infantry 

Fighting 

Vehicle 

Cavalry 

Fire 

Integrated 

Support 

Team 

Engineer 

Self- 

Propelled 

Artillery 

Mortar 

Carrier 

Command 
and  Control 

Med 

General 

Purpose 

M1A2  SEP 
Tusk 

M2A3 

OIF 

M3  A3 
OIF 

M3  A3 
BFIST 

M2A2  ODS-E 

M109A6 

Ml  064 A3 

Ml  06 8 A3 

Ml  13  A3 
Medical 

M113A3 

General 

Purpose 

M1ECP 

M2A3 ECP 

M3  A3  ECP 

M3  A3 
BFIST 
ECP 

M2A2  ODS-E 
ECP 

PIM 

Bradley 

Derivative 

Bradley 

Derivative 

Bradley 

Derivative 

Bradley 

Derivative 

M1ECPII 

M2A3 
ECP  II 

M3  A3 
ECP  II 

M3  A3 
BFIST 
ECP  II 

M2A2  ODS-E 
ECP  II 

PIM  ECP 

GCV 

Derivative 

GCV 

Derivative 

GCV 

Derivative 

GCV 

Derivative 

M1ECP  III 

M2A3 ECP 
III 

M3  A3  ECP 

III 

M3  A3 
BFIST 
ECP  III 

M2A2  ODS-E 
ECP  III 

MCV  OIF 

MRAP 

Caiman 

MRAP 

Caiman 

MRAP- 

M-ATV 

Ml  AMOD 

M2A3 

BMOD 

M3  A3 
BMOD 

M3  A3 
BFIST 
BMOD 

M2A3  (E) 
BMOD 

MCV  SWaP 
Recovery/ 
FP  Upgrade 

POC-V 

POC-V 

MRAP- 

MaxxPro 

GCV 

Turretless 

Bradley 

Armored 

Knight 

Turretless 

Bradley 

MCV  SMOD 

C2  OIF 

MEV-OIF 

ICV  OIF 

Turretless 

GCV 

Turretless 

Bradley 

Turretless 

GCV 

C2  SMOD 

MEV  SWaP 
Recovery/ 
FP  Upgrade 

ICV  SWaP 
Recovery/ 
FP  Upgrade 

Turretless 

GCV 

EVS  OIF 

MEV 

SMOD 

ICV  SMOD 

FSV  OIF 

ESV  SMOD 

FSV  SMOD 

|  SBCT  Roles  1 

Infantry 

Carrier 

Anti-Tank 

Guided 

Munitions 

Command 

and 

Control 

Reconnais¬ 

sance 

Mortar 

Carrier 

Fire 

Support 

Engineering 

Squad 

Medical 

Evacuation 

Nuclear, 
Biological 
&  Chemical 
Reconnais¬ 
sance 

Mobile  Gun 
System 

OIF 

OIF 

OIF 

OIF 

OIF 

OIF 

OIF 

OIF 

OIF 

OIF 

SWaP 
Recovery/ 
FP  Upgrade 

SWaP 
Recovery/ 
FP  Upgrade 

SWaP 
Recovery/ 
FP  Upgrade 

SWaP 
Recovery/ 
FP  Upgrade 

SWaP 

Recovery/ 

FP  Upgrade 

SWaP 
Recovery/ 
FP  Upgrade 

SWaP 

Recovery/ 

FP  Upgrade 

SWaP 

Recovery/ 

FP  Upgrade 

SWaP 
Recovery/ 
FP  Upgrade 

SWaP 
Recovery/ 
FP  Upgrade 

System 

Digitization 

System 

Digitization 

System 

Digitization 

System 

Digitization 

System 

Digitization 

System 

Digitization 

System 

Digitization 

System 

Digitization 

System 

Digitization 

System 

Digitization 

Full 

SMOD 

Full 

SMOD 

Full 

SMOD 

Full 

SMOD 

Full 

SMOD 

Full 

SMOD 

Full 

SMOD 

Full 

SMOD 

Full 

SMOD 

Full 

SMOD 

The  49  attributes  in  the  performance  model  were  selected  through  analysis  of  the 
most  recent  and  relevant  requirement  documents,  a  subject  matter  expert  (SME)  panel, 
and  input  from  AMSAA. 


1  The  analysis  was  conducted  on  49  attributes,  rather  than  the  full  52  in  the  hierarchy,  due  to  lack  of  available  data 
for  three  of  the  sustainability  attributes. 
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The  results  of  the  performance  evaluation  are  combined  with  the  cost  analysis  and 
scheduling  analysis  in  the  optimization  model.  The  optimization  model  embedded  in 
CP  AT  is  designed  to  provide  the  ability  to  understand  the  trade  space  between  cost, 
schedule,  and  perfonnance  in  order  to  assist  in  planning  for  the  overall  Combat  Vehicle 
portfolio  and  fleet  modernization. 

1.3  INTENDED  USE 

CP  AT  is  intended  to  identify  optimum  courses  of  action  (cost,  schedule,  and 
perfonnance)  for  the  PEO  GCS  portfolio  investment  (Edwards,  2011b). 

The  PEO  GCS  Operations  (OPS)  are  currently  executing  a  comprehensive 
assessment  of  their  modernization  initiatives.  CPAT  is  intended  to  provide  the  analytical 
underpinnings  that  support  an  achievable  and  affordable  Combat  Vehicle  Modernization 
Strategy. 

As  a  decision-making  tool,  CPAT  is  also  intended  to  provide  PEO  with  rapid 
assessments  of  “bang  for  buck”  alternative  questions.  CPAT  is  intended  to  support  the 
user  community  in  requirements  development  for  CBA-type  analysis. 

The  longer-term  objective  is  for  CPAT  to  be  utilized  for  annual  updates  and  to 
support  PMs,  PEO,  and  DA  in  making  future  investment  decisions.  It  is  intended  that  the 
Optimization  Model’s  Graphical  User  Interface  (GUI)  will  help  the  PM  analyze  possible 
future  “what  if’  scenarios.  In  the  future,  it  is  also  intended  that  PEO  GCS  will  continue  to 
develop  the  tool  and  conduct  detailed  data  collection  using  the  same  methodology  in 
support  of  the  PMs’  technology  trade  assessments  (Edwards,  2011b). 

1.4  CPAT  ASSUMPTIONS 

For  the  development  of  CPAT,  the  CPAT  team  outlined  the  following  list  of 
assumptions  (Edwards,  2011b): 

1.  The  timeframe  of  analysis  is  2013-2040 

2.  Will  modernize  its  fleet 

3.  HBCT  and  SBCT  will  maintain  current  objective  table  of  organization  and 
equipment  (OTOE)  levels — but  these  can  be  integrated 

4.  There  will  be  two  brigade  types:  (1)  HBCT  with  24  brigades  and  10 
missions;  each  can  be  upgraded  separately;  (2)  SBCT  with  9  brigades. 
SBCT  must  be  upgraded  in  entire  brigade  quantities.  As  a  result,  multiple 
missions  are  modeled  as  a  single  SBCT  mission 

5.  The  alternatives  for  each  mission  role  are  the  same  as  the  alternatives  in 
the  upcoming  AoA 

6.  In  the  perfonnance  evaluation  it  is  assumed  that  the  attributes  selected  are 
robust  enough  to  provide  trade  space  between  alternatives 

7.  Performance  evaluation  value  functions  were  derived  using  Full  Spectrum 
Operation 

8.  Field  schedules  are  fixed  inputs 
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9.  The  minimum  sustaining  rate  for  production  is  one  Brigade  per  year 

10.  Modernization  plans  are  limited  to  what  is  currently  proposed  by  the 

individual  PMs 

11.  Ml  13  must  be  divested  from  fleet  by  2030 

12.  When  production  ceases  on  the  modernization  program  the  line  will 

not  restart 

1 3 .  Cost  Assumptions — 

a.  Base  year  of  analysis  is  2012 

b.  Period  of  comparison  is  FY2013-FY2028 

c.  Weapon  system  populations  are  limited  to  Brigade  Combat  Team 
assigned  assets,  Operational  Readiness  Floats  (ORFs),  TRADOC 
assets  and  Research  Facilitation  Team  (RFT)  assets 

d.  Starting  weapon  system  populations  within  each  BCT  mission  role 
will  remain  constant 

e.  Only  modification  and  replacement  configurations  as  defined  in 
the  systems  book  will  be  studied 

f.  Costs  do  not  account  for  schedule  delays  or  Industrial  Base  Office 
(IBO)  uncertainty 

g.  RDTE  expenditures  prior  to  FY2013  are  not  included 

h.  Modified  or  replacement  systems  will  be  fielded  to  Active  Army 
brigades  followed  by  Prepositioned  Stock  and  Reserve/National 
Guard  units 

i.  Costs  are  modeled  assuming  a  Peacetime  OPTEMPO 

j.  Required  RDTE  prior  to  FY2013  will  be  completed  for  all 
alternatives 

1.5  CPAT  LIMITATIONS 

The  CPAT  team  identified  several  current  limitations.  First,  in  the  area  of 
performance  evaluation,  only  a  single  SME-user  panel  has  been  conducted.  Therefore,  it 
is  recommended  that  the  team  conduct  an  additional  two  panels.  Second,  the  perfonnance 
evaluation  does  not  account  for  the  synergistic  effects  of  technology.  Third,  the  cost 
analysis  does  not  account  for  step-function  average  per  unit  cost  (APUC) — this  is 
scheduled  to  be  implemented  in  FY2012.  The  cost  analysis  also  does  not  account  for 
military  personnel  (MILPRS)  and  military  construction  (MILCON).  Fourth,  the  schedule 
does  not  penalize  cost  or  perfonnance  for  schedule  slip  (Edwards,  2011b). 
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2.  CPAT  REQUIREMENTS 


2.1  GENERAL  REQUIREMENTS 

At  the  highest  level,  CPAT  is  required  to  fulfill  the  following  requirements: 

1.  Identify  optimal  courses  of  action  for  PEO  GCS  fleet  investment 

2.  Be  analytically  credible  and  repeatable 

3.  Be  auditable  and  transparent 

4.  Include  holistic  perspective  of  key  stakeholders:  analytical  community, 
headquarters,  OSD 

2.2  MODELING  REQUIREMENTS 

The  following  lists  the  CPAT  modeling  requirements: 

1.  Trained  analysts  (Sandia  currently)  must  run  the  model  to  detennine 
parameter  sensitivity  and  typical  output  requirements  for  development 

2.  Very  large  input  performance  data  requirements;  -620+  attributes 

3.  Software  installation  is  self-contained  and  does  not  require  external  load 
of  CPLEX  or  Open  Programming  Language  (OPL) 

4.  Standardized  output  and  GUI  to  allow  for  PM  use  of  CPAT  for 
final  version 

5.  Consider  for  future  development  to  other  Anny  equipment  portfolio 
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3. 


cpat  v&y 


3.1  V&V  SCOPE 

The  purpose  of  this  technical  review  is  to  determine  whether  CPAT  perfonns  its 
intended  functions  correctly  and,  to  a  lesser  extent  where  possible,  to  ensure  that  CPAT 
perfonns  no  unintended  functions  and  to  measure  its  quality  of  solutions.  The  verification 
of  CPAT  focuses  on  the  performance  model  structure,  the  formulation  of  CPAT,  and  the 
OPL  implementation.  The  verification  includes  an  evaluation  of  documents,  code, 
requirements,  and  specifications.  The  validation  of  CPAT  investigates  the  quality  of  the 
solutions  represented  for  the  data  given  to  test  the  robustness  of  the  model  to  small 
changes  in  the  model’s  parameters,  and  to  investigate  the  correctness  of  the  solutions 
given.  We  do  not  conduct  exhaustive  sensitivity  analysis  by  trying  to  duplicate  what  the 
sponsor  has  already  done,  nor  do  we  attempt  to  evaluate  the  data  given  or  the  “qualitative” 
correctness  of  the  solution  from  a  policy  perspective. 

To  conduct  a  review  of  the  CPAT  model  we  use  the  database, 
Unrestricted_4.0B_b30.cpat.  The  technical  review  also  utilizes  the  following  supplied 
documentation: 

1.  L.  Andrade,  G.  Kao,  C.  Lawton,  D.  Melander,  and  R.  Rice.  Working 
Paper.  Fleet  Management  Planning  Decision  Analysis:  A  Mixed  Integer 
Linear  Program  Formulation 

2.  Program  Executive  Office  (PEO)  Ground  Combat  System  (GCS).  “PEO 
GCS  Capability  Portfolio  Analysis  Tool  (CPAT).”  August  23,  2011 

3.  PEO  GCS.  “PEO  GCS  Capability  Portfolio  Analysis  Tool  (CPAT).” 
November  15,  201 1 

4.  PEO  GCS.  “PEO  GCS  Capability  Portfolio  Analysis  Tool  (CPAT).” 
December  9,  201 1 

5.  Sandia  National  Laboratories.  “CPAT  Sensitivity  Analysis.” 
November  29,  2011 

6.  Sandia  National  Laboratories.  “CPAT  Objective  Function  Studies.” 
November  30,  2011 
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4.  y&V  OF  THE  VALUE  MODEL 


4.1  PERFORMANCE  MODEL  OVERVIEW 

The  CP  AT  performance  evaluation  uses  a  Multiple  Objective  Decision  Analysis 
(MODA)  approach  for  assessing  the  value  of  vehicle  modernization  in  the  HBCT  and 
SBCT  combat  fleets.  The  MODA  approach  provides  insight  to  decision  makers  who  are 
faced  with  a  problem  that  has  multiple  competing  objectives.  Figure  2  shows  the  general 
qualitative  value  hierarchy  framework  that  the  CPAT  team  used  to  decompose  the 
problem  into  the  subobjectives  concerning  each  role’s  objective.  Every  subobjective  has 
a  set  of  attributes  that  measure  the  achievement  of  a  subobjective’s  performance  intent. 
Value  functions  are  used  to  transform  the  raw  vehicle  performance  data  from  each 
attribute  into  a  normalized  scale  ranging  from  0  to  1 ,  and  to  measure  the  returns  to  scale 
for  a  given  attribute. 


Role 

Objective 


Subobjectives 

Subobjective  1 

i 

Subobjective  2 

Attributes  Attribute  1  Attribute  2  Attribute  3  Attribute  4  Attribute  5 


Vehicle  Performance  Data 


Figure  2.  The  CPAT  Value  Hierarchy  decomposes  the  problem  into  subobjectives 
concerning  each  role.  Attributes  indicate  each  sub  objective’s  perfonnance  and 
value  functions  are  used  to  measure  the  returns  of  scale  for  a  given  attribute. 

The  MODA  approach  promotes  buy-in  from  multiple  stakeholders.  The  CPAT 
team  held  an  SME  panel  with  key  stakeholders  in  Fort  Benning,  Georgia,  from 
November  30  to  December  2,  2010.  The  SME  panel  consisted  of  16  individuals  drawn 
primarily  from  the  MCoE  TRADOC  Capability  Manager  (TCM)  Heavy  Brigade  Combat 
Team  (HBCT),  TCM  Battlefield  Surveillance  Brigade  (BFSB),  TCM  Stryker  Brigade 
Combat  Team  (SBCT),  along  with  representatives  from  FCoE,  CASCOM,  and  AMEDD. 
The  panel  focused  on  refining  the  attributes  and  their  trade  space  bounds,  generating 
value  functions  for  each  of  the  attributes,  developing  weights  using  the  swing  weight 


2  NPS  Operations  Research  analysts  have  adapted  the  terminology  in  the  hierarch  slightly  changing  major  attributes 
to  sub-objective,  measures  to  attributes,  and  value  functions  to  measures  for  our  analysis. 
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matrix  technique,  and  establishing  role  weights  using  pair-wise  comparisons 
(Edwards,  2011b). 


4.2  PERFORMANCE  MODEL  CONSTRAINTS,  LIMITATIONS,  AND 
ASSUMPTIONS 

In  order  to  adhere  to  the  timeline,  the  attributes  were  selected  using  as  much 
existing  data  and  studies  as  possible  (see  Table  2  for  attribute  data  sources). 

Table  2.  CP  AT  data  sources. 


Role 

Source  Documents 

Direct  Fire  Infantry  Fighting  Vehicle  (IFV);  Fire 
Integrated  Support  Team  (FIST);  Engineer  (Eng); 
Cavalry  (CAV) 

BFV  FOV  CDD  Block  11  -  16  Apr  2010 

GCV  FOV  CDD  -  10  Mar  2010 

Indirect  Fire  Self-Propelled  Artillery  (SPA) 

Ml 09  FOV  CPD 

Mortar;  C2  Vehicle;  Medical;  General  Purpose 

AMPV  CDD  31  AUG  10;  (w/INC  1,  2) 

M113A3  Specification 

MRAP  FOV  CPD  vl.l  Draft  2009-04-10 

Direct  Fire  Main  Battle  Tank  (MBT) 

Abrams  CDD  1  JUN  10 

Stryker  Family  of  Vehicles  (FOV) 

Stryker  ORD  (w/Block  1,  2,  3) 

The  performance  model  has  several  key  limitations: 

1.  The  value  model  does  not  capture  potential  performance  synergies 
across  roles  in  the  overall  perfonnance  of  a  vehicle.  For  example,  the 
added  value  of  a  better  thermal  sight  for  an  IFV  may  also  affect  other 
roles  in  the  same  fonnation  (Edwards,  2011b). 

2.  The  value  model  does  not  account  for  potential  performance  synergies 
based  on  vehicle  quantities  in  each  role.  Meaning  each  vehicle  in  a 
type  is  assumed  to  provide  the  same  performance  (Edwards,  2011b). 

3.  Trade-offs  among  the  roles  is  not  considered,  which  results  in  20 
separate  performance  models. 

The  primary  perfonnance  evaluation  assumption  is  that  all  vehicles  in  a  role  are 
configured  similarly  and  role  weightings  represent  trade-offs  in  the  full  spectrum 
of  operations. 

4.3  CHARACTERISTICS  OF  THE  CPAT  PERFORMANCE  MODEL 

4.3.1  CPAT  Value  Hierarchy 

The  CPAT  team  uses  a  value  hierarchy  to  assess  the  value  of  each  vehicle 
alternative  within  each  role.  A  total  of  47  vehicles  are  analyzed  by  CPAT.  Each  of  the  20 
roles  has  its  own  hierarchy  composed  of  the  six  subobjectives:  survivability,  growth, 
lethality,  personnel  payload,  mobility,  and  sustainability.  Each  subobjective  has  a  unique 
set  of  attributes  that  collectively  assess  a  vehicle’s  value.  Table  3  shows  the  CPAT 
Qualitative  Value  Hierarchy  with  the  49  possible  attributes  used  within  each  role.  Some 
of  the  attributes  in  value  hierarchy  can  be  grouped  such  as  “Occupant  Protection”  against 
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Small  Anns  and  Medium  Caliber,  Rocket  Propelled  Granade  (RPG),  Anti-Tank  Guided 
Missile  (ATGM),  Tank  KE,  Top  Attack,  Undervehicle,  and  Improvised  Explosive  Device 
(IED).  We  refer  to  this  grouping  of  “subattributes”  as  categories.  The  same  is  true  for  the 
survivability  attribute.  We  classify  the  following  as  categories  of  “Survivability”  against 
Small  Anns  and  Medium  Caliber,  RPG,  ATGM,  Top  Attack,  Undervehicle,  IED.  These 
categories  are  reflected  in  Table  3. 

Table  3.  CP  AT  Qualitative  Performance  Model  below  is  represented  by  the  six 

primary  subobjectives  in  the  columns  and  49  attributes  shown  in  the  rows.  For 
example,  the  survivability  objective  has  four  attributes:  Occupant  Protection, 
CBRN  Operation,  Survivability,  and  Silent  Watch.  The  Occupant  Protection  and 
Survivability  attributes  are  each  described  by  seven  category  measures. 


Survivability 

Growth 

Lethality 

Personnel 

Payload 

Mobility 

Sustainability 

Protection  against 
Small  Arms  & 
Medium  Caliber 
(Category) 

Available 

Electrical 

Power  (Linear) 

Effective  Range 
vs.  Tanks 
(Linear) 

Personnel 

Carry 

(Linear) 

Soft  Soil  Mobility 
(Qualitative) 

Operational 
Availability  -  Initial 
(Linear) 

Protection  against 
RPG  (Category) 

Exportable 

Electrical 

Power  (Linear) 

Effective  Range 
vs.  IFVs  (Linear) 

Litter  Carry 
(Linear) 

Max  Speed 
(Linear) 

Operational 
Availability  -  Worst 
Case  (Linear) 

Protection  against 
ATGM  (Category) 

Information 
System  Growth 
(Qualitative) 

Effective  Range 
vs.  Light  Armor 
(Linear) 

Turning  Diameter 
(Linear) 

Sustainment 
Availability  (Linear) 

Occupant 

Protection 

(Qualitative) 

Protection  against 
Tank  KE 
(Category) 

Effective  Range 
vs.  Personnel 
(point  target) 
(Linear) 

Static  Stability 

Factor 

(Qualitative) 

Port  Commonality* 
(Linear) 

Protection  against 
Top  Attack 
(Category) 

Effective  Range 
vs.  Personnel 
(area  target) 
(Linear) 

Vehicle  Width 
(Linear) 

Embedded 

Diagnostics* 

(Linear) 

Protection  against 

Undervehicle 

(Category) 

Efficiency  vs.  Air 
Defense  Artillery 
(ADA)  site 
(Linear) 

Dash  Speed 
(Linear) 

Mean  Time 

Between  Essential 
Function  and 

Failure*  (Linear) 

Protection  against 
IED  (Category) 

Efficiency  vs. 
Artillery  Platoon 
(Linear) 

Air 

Transportability 

(Qualitative) 

Chemical,  Biological,  Radiological, 
and  Nuclear  (CBRN)  Operation 
(Qualitative) 

Efficiency  vs. 
Mech  Platoon 
(Linear) 

Cruising  Range 
(Linear) 

Survivability 
against  Small  Arms 
&  Medium  Caliber 
(Category) 

Efficiency  vs. 
Multiple  Rocket 
Launcher  (MRL) 
(Linear) 

Survivability 

(Qualitative) 

Survivability 
against  RPG 
(Category) 

Efficiency  vs. 
Dismounted  Crew 
(Linear) 

Survivability 
against  ATGM 
(Category) 

Missile 

Capability 

(Qualitative) 

Survivability 
against  Tank  KE 
(Category) 

ATGM  Fire 
Control  (Binary) 
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Survivability 

Growth 

Lethality 

Personnel 

Payload 

Mobility 

Sustainability 

Survivability 
against  Top  Attack 
(Category) 

Mast  Mounted 
Sensor  (Binary) 

Survivability 

against 

Undervehicle 

(Category) 

Target 

Designation 

(Binary) 

Survivability 
against  IED 
(Category) 

Far  Target 
Location  (FTL) 
Targeting 
(Binary) 

Silent  Watch  (Linear) 

Weapon 

Stabilization 

(Binary) 

Transition  from 
Firing  to  Moving 
(Linear) 

(*)  Data  currently  unavailable  for  these  attributes;  they  are  not  used  in  the  analysis. 


Many  of  the  attributes  are  the  same  across  all  of  the  qualitative  models,  but  in 
some  instances  not  all  of  the  quantitative  characteristics  of  the  attributes  are  identical. 
Table  4  provides  a  breakdown  of  the  number  of  attributes  used  in  each  role  hierarchy  as 
well  as  the  mean,  median,  and  standard  deviation.  The  number  of  attributes  used  in  each 
of  the  20  role  value  hierarchies  ranges  from  31  to  41.  When  considering  all  roles,  the 
average  number  of  attributes  used  is  37.1,  with  a  standard  deviation  of  3.5. 

Table  4.  Number  of  attributes  used  for  HBCT  and  SBCT  roles. 


H 

1BC1 

r 

Role 

MBT 

IFV 

CAV 

FIST 

Eng 

SPA 

MtrCr 

C2  (H) 

Med 

GP 

No.  Attributes 

38 

40 

40 

40 

37 

36 

35 

31 

31 

31 

Mean 

35.9 

Median 

36.5 

Standard  Deviation 

3.8 

SBC! 

Role 

ICV 

ATGM 

C2  (S) 

RV 

MCV 

FSV 

ESV 

MEV 

NBCRV 

MGS 

No.  Attributes 

41 

41 

35 

39 

39 

39 

41 

34 

34 

40 

Mean 

38.3 

Median 

39.0 

Standard  Deviation 

2.9 

HBCT/SBCT 

Mean 

37.1 

Median 

38.5 

Stand  Deviation 

3.5 

Thirty  of  the  49  attributes  are  used  by  all  20  roles,  which  include  all  attributes  in 
the  Survivability,  Growth,  Mobility,  and  Sustainability  subobjectives.  However,  nine 
attributes  are  used  in  four  or  fewer  of  the  qualitative  value  hierarchies.  These  include 
Transition  from  Firing  to  Moving  (used  in  one  role),  Litter  Carry  (two  roles),  Efficiency 
versus  ADA  site,  Efficiency  versus  artillery  platoon,  Efficiency  versus  Mech  Platoon, 
Efficiency  versus  MRL,  Efficiency  versus  Dismounted  Crew,  ATGM  Fire  Control  (three 
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roles),  and  Mast  Mounted  Sensor  (four  roles).  The  Descriptive  Statistics  Summary  Table 
in  Appendix  A  displays  the  attribute-role  mapping. 

4.3.2  Value  Functions 

The  model  uses  value  functions  to  transform  the  raw  vehicle  data  from  each 
attribute  into  a  nonnalized  scale  ranging  from  0  to  1.  The  shape  of  the  value  function 
defines  the  returns  of  scale  for  a  particular  attribute. 

Each  attribute  has  a  value  function  associated  with  it.  The  hierarchy  includes  a 
mix  of  20  attributes  with  constructed  scales,  24  attributes  that  have  piecewise  linear 
scales  with  at  least  one  inflection  point,  and  5  attributes  that  have  binary  (yes/no)  scales. 
The  constructed  scales  consist  of  bins  that  indicate  the  lowest  to  highest  impact  levels. 
For  example,  the  Information  System  Growth  attribute  has  a  constructed  scale  with  bins 
ranging  from  none  (low)  to  a  combination  of  radio  suites  (high). 

4.3.3  Weighting  the  Attributes 

Attribute  weights  assess  the  trade-off  among  the  different  subobjectives  within  a 
role.  A  significant  finding  of  this  review  is  that  attributes  within  a  role  were  assessed 
using  the  swing  weight  matrix.  However,  attributes  across  roles  where  not  assessed. 
Therefore,  it  is  not  advised  to  compare  vehicle  performance  values  outside  a  given  role 
because  the  trade-offs  among  the  different  roles  have  not  been  determined. 

Each  of  the  49  possible  attributes  within  the  CP  AT  value  hierarchy  is  assessed  a 
weight  with  a  possible  range  from  0  to  1 .  The  sum  of  all  weights  within  each  qualitative 
hierarchy  is  equal  to  1.  The  maximum  weight  recorded  for  an  attribute  is  0.055,  with 
90%  of  the  attributes  across  all  roles  ranging  from  0  to  0.039.  The  average  weight  across 
all  attributes  is  0.027. 

The  structure  of  the  qualitative  value  model  is  additive  by  design.  Therefore,  the 
weights  for  all  attributes  supporting  a  given  objective  are  additive  and  when  aggregated, 
the  subobjective  relative  weights  are  obtained.  The  weights  for  each  of  the  six 
subobjectives  across  the  20  roles  are  shown  in  Figure  3. 


3  A  weight  of  0.055  is  used  for  seven  attributes  including  Occupant  Protection  against  Small  Anns  &  Medium 
Caliber,  Occupant  Protection  against  IED,  Survivability  against  Small  Arms  &  Medium  Caliber,  Cruising  Range, 
Survivability  against  Undervehicle,  Survivability  against  IED,  and  Soft  Soil  Mobility. 


15 


HBCT 

MBT 

IFV 

CAV 

FIST 

Eng 

SPA 

MtrCr 

C2  (H) 

Med 

GP 

Survivability 

0.488 

0.432 

0.411 

0.464 

0.499 

0.434 

0.454 

0.518 

0.489 

0.548 

Growth 

0.068 

0.072 

0.070 

0.075 

0.086 

0.089 

0.060 

0.146 

0.112 

0.042 

Lethality 

0.211 

0.225 

0.252 

0.207 

0.053 

0.205 

0.185 

0.000 

0.000 

0.000 

Personnel  Payload 

0.000 

0.031 

0.032 

0.007 

0.039 

0.000 

0.000 

0.026 

0.044 

0.032 

Mobility 

0.151 

0.162 

0.151 

0.164 

0.219 

0.180 

0.190 

0.193 

0.235 

0.250 

Sustainability 

0.081 

0.078 

0.083 

0.083 

0.104 

0.091 

0.111 

0.117 

0.119 

0.127 

SBCT 

ICV 

ATGM 

C2  (S) 

RV 

MCV 

FSV 

ESV 

MEV 

NBCRV 

MGS 

Survivability 

0.394 

0.411 

0.469 

0.395 

0.410 

0.398 

0.383 

0.481 

0.504 

0.399 

Growth 

0.074 

0.077 

0.115 

0.094 

0.076 

0.096 

0.072 

0.125 

0.094 

0.064 

Lethality 

0.247 

0.248 

0.078 

0.211 

0.219 

0.206 

0.239 

0.042 

0.073 

0.247 

Personnel  Payload 

0.034 

0.014 

0.039 

0.034 

0.032 

0.034 

0.034 

0.036 

0.010 

0.010 

Mobility 

0.165 

0.163 

0.193 

0.178 

0.169 

0.175 

0.180 

0.201 

0.211 

0.190 

Sustainability 

0.086 

0.088 

0.105 

0.087 

0.094 

0.091 

0.091 

0.115 

0.107 

0.090 

Figure  3.  Subobjective  weights  for  each  FIBCT  and  SBCT  role. 

Note:  Highest  overall  weight  is  shown  in  green,  the  lowest  overall  weights  are  shown  in  red,  and 
nonweighted  subohjec fives  are  shown  in  gray. 

The  Survivability  subobjective  makes  up  approximately  55%  (shown  in  green)  of 
the  total  weight  for  the  GP  role,  while  Lethality  receives  no  weight  and  Personnel 
Payload  receives  only  3.2%  of  the  total  weight  for  the  GP  role.  For  the  FIST  role, 
Survivability  receives  only  46%  of  the  weight  and  provides  less  the  0.7%  of  its  total 
weight  toward  Personnel  Payload  (shown  in  red).  In  general,  Survivability  is  weighted 
the  highest  across  all  roles  and  Personnel  Payload  the  lowest  across  all  roles  (see 
Figure  4).  The  weights  for  Growth  and  Sustainability  range  between  4.2%  and  14.6%  for 
all  roles,  and  the  Mobility  subobjective  weights  range  from  15.1%  to  25%  for  all  roles. 
The  Lethality  weights  range  from  4.2%  to  25.2%  for  all  roles  except  C2(H),  Med,  and 
GP,  which  receive  no  weight  for  the  underlying  attributes.  The  Personnel  Payload 
weights  range  from  0.7%  to  4.4%  with  no  weights  for  the  MBT,  SPA,  and  MtrCr  roles. 
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Weights  of  Subobjectives  for  all  20  Roles 


♦  Survivability  ■  Growth  a  Lethality  Personnel  Payload  l  Mobility  •  Sustainability 


Figure  4.  Weighting  of  Subobjectives  for  all  20  roles.  Survivability  is  weighted  the 
highest  for  each  role  and  Personnel  Payload  is  the  lowest  in  all  but  three  roles. 

We  did  not  attempt  to  make  a  judgment  based  on  the  resulting  subobjective 
weights.  However,  it  is  important  for  leadership  and  users  of  CPAT  to  understand  the 
underlying  emphasis  a  given  role  places  on  a  given  subobjective  in  terms  of  the  value 
trade-offs.  Therefore,  a  subject  matter  expert  review  should  ensure  that  the  weight  for  a 
given  subobjective  and  for  a  given  role  is  verified.  If  it  is  found  later  that  the  weights  for 
the  subobjectives  require  an  adjustment  then  we  suggest  a  revision  beginning  with  the 
value  model’s  subobjectives  and  supporting  attributes,  followed  by  a  reassessment  of  the 
attribute  weights. 

4.4  PERFORMANCE  MODEL  VALIDATION 

For  the  performance  model  validation,  we  examine  the  impact  of  changes  to  the 
major  model  parameters  to  evaluate  robustness.  Specifically,  we  examine  how  changes  to 
swing  weights  and  value  function  shape  affect  alternative  selection.  We  also  look  at  how 
influential  a  value  function  is  in  terms  of  evaluating  the  return  to  scale  of  an  attribute. 

4.4.1  Swing- Weight  Sensitivity  Analysis 

We  use  one-way  sensitivity  analysis  to  detennine  the  impact  of  changes  in  the 
weights  on  the  ranking  of  vehicles.  Ideally,  we  would  perform  one-way  sensitivity 
analysis  on  all  possible  49  attributes  used  in  the  value  model;  however,  because  most 
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attributes  represent  less  than  3%  of  the  total  trade  space  available,  changing  only  one 
attribute  at  a  time  would  likely  not  impact  the  results.  Therefore,  we  opt  to  conduct 
changes  of  the  weights  for  each  of  the  six  subobjectives,  one  objective  at  a  time. 

The  sensitivity  analysis  is  conducted  for  all  47  vehicles  within  the  20  roles  to  test 
the  robustness  of  the  CP  AT  model.  It  is  important  to  remind  the  reader  that  only  a  small 
number  of  alternatives  are  considered  in  each  role,  typically  only  four  to  six  vehicles.  The 
weights  for  each  role  in  the  green  (unclassified)  and  red  (classified)  versions  of  the  model 
are  the  same.  Therefore,  the  analysis  performed  on  the  green  (unclassified)  model  is 
representative  of  results,  which  would  be  derived  from  the  red  model. 

Figure  5  illustrates  the  sensitivity  analysis  conducted  for  the  seven  vehicles 
considered  within  the  CAV  role.  The  seven  lines  on  the  graph  show  how  each  vehicle’s 
performance  changes  as  we  vary  the  weight  on  the  lethality  objective  from  0  to  1.  When 
lethality  is  weighted  from  0.0  to  0.650  the  Turretless  GCV  is  selected  as  the  highest  value 
alternative.  At  0.650  the  highest-value  alternative  changes  to  the  M3A3  BMOD.  We 
define  this  inflection  point  as  a  breakpoint.  The  graph  also  shows  the  distance  between 
the  originally  assigned  weight  and  the  breakpoint.  If  the  distance  between  the  assigned 
weight  and  the  breakpoint  is  large,  the  weight  is  not  sensitive  to  change,  but  a  weight  that 
is  close  to  the  breakpoint  is  sensitive  to  change.  For  the  Lethality  subobjective  in  the 
CAV  role,  the  assigned  weight  is  0.252  and  the  distance  to  the  nearest  breakpoint  0.398. 
In  other  words,  it  would  take  an  increase  of  over  150%  in  the  underlying  attributes 
supporting  the  lethality  subobjective  before  the  model  would  select  a  different  alternative. 
In  this  case,  we  say  that  the  Lethality  subobjective  for  the  CAV  role  is  very  insensitive  to 
changes  in  the  17  attributes  that  add  up  to  the  Lethality  subobjective  weight. 


Sensitivity  Analysis  for  Lethality  (CAV  Role) 


- M3  A  3  OIF 

- M3  A  3  ECP 

- M3A3  ECP  II 

- M3A3  ECP  III 

- M3  A  3  BMOD 

- TuiTetless  Bradley 

- Turretless  GCV 

Weight 


Figure  5.  Sensitivity  Analysis  conducted  for  the  seven  vehicles  considered  within 
the  CAV  role  illustrating  the  breakpoint.  Turretless  GCV  is  the  highest-value 
alternative  when  lethality  is  weighted  from  0.0  to  0.650.  For  weights  greater  than 
0.650,  the  highest-value  alternative  changes  to  the  M3  A3  BMOD. 
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In  general,  the  results  of  the  sensitivity  analysis  suggest  that  the  value  model  is 
very  robust  to  changes  in  attribute  swing- weights.  Table  5  illustrates  the  percentage 
weight  change  required  before  the  model  would  select  a  different  alternative.  Cells  with  a 
dash  (-)  do  not  have  a  breakpoint.  The  cells  highlighted  in  red  show  that  the  lowest- 
percentage  weight  change  required  to  alter  the  preferred  vehicle  alternative  is  76%  for  the 
Lethality  subobjective  in  the  MBT  role  and  the  Survivability  subobjective  for  the  CAV 
role.  All  other  subobjectives  across  the  20  HBCT  roles  require  greater  than  a  100% 
change  in  weight  for  the  model  to  select  a  different  alternative. 

Table  5.  Percent  increase  change  in  swing- weight  required  for  change  in  alternative 
selection  (assuming  one  change  at  a  time  at  the  objective  level).  A  dash  indicates 
that  no  change  can  be  made  for  the  role  and  objective  indicated. 


HBCT 

MBT 

1 IV 

CAV 

FIST 

Eng 

SPA 

MtrCr 

C2(H) 

Med 

GP 

Survivability 

- 

- 

76% 

- 

- 

- 

- 

- 

- 

91% 

Growth 

923% 

- 

1257% 

- 

- 

- 

- 

- 

- 

- 

Lethality 

76% 

- 

158% 

286% 

- 

- 

- 

- 

- 

- 

Personnel  Payload 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

Mobility 

- 

395% 

- 

419% 

- 

- 

348% 

263% 

219% 

220% 

Sustainability 

- 

- 

- 

801% 

- 

- 

573% 

585% 

572% 

- 

SECT 

ICV  ATGM  C2  (S)  RV  MCV  FSV  ESV  MEV  NBCRV  MGS 

Survivability 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

Growth 

- 

- 

- 

- 

816% 

524% 

1 145% 

181% 

648% 

369% 

Lethality 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

Personnel  Payload 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

Mobility 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

Sustainability 

- 

- 

658% 

990% 

- 

938% 

- 

- 

- 

- 

4.4.2  Value  Function  Relevance  Analysis 

Ideally,  we  want  the  quantitative  value  functions  of  a  model  to  have  a  scale  that  is 
just  large  enough  to  contain  the  minimum  and  the  maximum  score  of  the  alternatives 
being  measured,  and  no  more.  In  practice,  this  may  be  impossible  for  a  variety  of  reasons. 
For  example,  many  of  the  value  functions  for  the  CP  AT  performance  model  were 
assessed  before  the  ranges  of  the  actual  data  were  known.  When  time  is  available,  it  is 
ideal  to  go  back  and  reassess  the  value  function  for  the  “actual”  range  of  the  data,  but  this 
was  not  possible  with  the  current  version  of  CP  AT.  As  a  result,  it  is  possible  to  have  a 
model  with  many  measures  that  are  not  influential  or  do  not  discriminate  between 
alternatives.  In  other  words,  we  would  describe  these  value  measures  as  nonrelevant  to  a 
particular  decision  context.  The  intent  of  this  section  is  to  identify  those  measures  that  do 
not  utilize  the  full  scale  range  and  also  those  whose  alternatives  all  receive  the  same 
value  because  there  is  no  variation  across  the  alternatives  for  a  given  role  (or  across  roles, 
in  some  cases).  The  identified  measures  are  prime  candidates  for  elimination  with  the 
intent  of  achieving  a  more  parsimonious  model. 

For  this  analysis,  we  define  the  possible  range  as  the  minimum  and  maximum  on 
the  value  function  scale  and  the  actual  range  as  the  minimum  and  maximum  values  of 
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the  data  that  is  input  into  the  model.  For  example,  the  possible  range  used  for  the  Silent 
Watch  attribute  is  the  same  for  all  20  roles  (0  to  12  hours).  In  fact,  the  possible  range  is 
the  same  across  all  20  roles  for  27  of  the  49  attributes.  See  Appendix  B,  Attribute 
Summary  Table,  for  more  information  on  range  variation  among  attributes. 

Our  analysis  shows  that  the  actual  range  of  the  data  often  does  not  utilize  the  full 
possible  range  of  the  value  functions.  For  example,  Figure  6  shows  that  the  actual  range 
of  the  data  for  Max  Speed  in  the  CAV  role  utilizes  just  2%  of  the  possible  range.  It  is 
important  to  note  here  that  the  function  is  piecewise  linear,  but  the  data  will  only  show 
constant  returns  to  scale  for  value  increments  of  the  same  size  over  the  actual  range.  This 
function  may  not  be  measuring  at  the  correct  level  of  resolution  and  capturing  the  correct 
returns  of  scale,  and,  as  a  result,  the  measure  does  not  discriminate  properly  among 
alternatives  in  the  CAV  role. 


Max  Speed  (CAV  Role) 


Mile  Per  Hour 


Figure  6.  Max  Speed  (CAV  Role)  utilizes  just  2%  of  the  possible  range. 

For  the  24  quantitative  attributes  used  in  the  model,  only  10%  of  the  data’s  actual 
range  utilizes  more  than  50%  of  the  possible  value  function  range.  Approximately  half 
(51%)  of  the  value  function  data  use  0%  of  the  range,  which  means  there  is  no  variability 
and  all  vehicle  data  within  the  role  are  the  same.  This  happens  when  all  alternatives  score 
the  same  for  a  given  attribute  in  a  role  or  the  attribute  is  not  used  for  a  particular  role. 
Additionally,  7%  of  the  value  function  data  extend  beyond  the  established  range.  See 
Appendix  C  for  a  table  of  the  percent  range  used  for  all  quantitative  attributes. 

In  addition  to  the  quantitative  measures,  there  are  five  binary  attributes  in  the 
model.  The  binary  attributes,  however,  are  not  considered  for  all  vehicles  within  the 
SBCT/HBCT  roles.  For  example,  Target  Destination  is  considered  for  four  vehicle  types 
within  the  FIBCT  role  and  seven  vehicle  types  within  the  SBCT  role.  The  highlighted 
portions  of  Tables  6  and  7  show  where  100%  of  the  data  is  either  yes  or  no  for  a  given 
binary  attribute.  For  the  binary  measures,  we  want  to  identify  those  that  have  little  or  no 
influence  on  the  outcomes.  For  example,  the  Target  Designation  attribute  provides  a  nice 
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discriminatory  effect  among  HBCT  roles,  but  little  effect  on  outcomes  for  SBCT  roles. 
The  Mast-Mounted  Sensor  attribute,  however,  provides  no  value  for  any  attributes  in  any 
HBCT  role  and  very  little  in  the  SBCT  role. 


Table  6.  HBCT  Binary  Attribute  -  Percentage  of  Yes/No  for  each  role. 


HBCT 

Attribute 

MBT 

IFV 

CAV 

FIST 

ATGMFire  Control 

Yes 

- 

- 

- 

- 

No 

- 

- 

- 

- 

Mast-Mounted  Sensor 

Yes 

- 

- 

0% 

0% 

No 

- 

- 

100% 

100% 

Target  Designation 

Yes 

40% 

33% 

14% 

100% 

No 

60% 

67% 

86% 

0% 

FTL  Targeting 

Yes 

40% 

0% 

0% 

20% 

No 

60% 

100% 

100% 

80% 

Weapon  Stabilization 

Yes 

100% 

100% 

- 

- 

No 

0% 

0% 

- 

- 

Table  7.  SBCT  Binary  Attribute  -  Percentage  of  Yes/No  for  each  role. 


SBCT 

Attribute 

ICV 

ATGM 

C2  (S) 

RV 

MCV 

FSV 

ESV 

MEV 

NBCRV 

MGS 

ATGM  Fire  Control 

Yes 

0% 

25% 

- 

- 

- 

- 

0% 

- 

- 

- 

No 

100% 

75% 

- 

- 

- 

- 

100% 

- 

- 

- 

Mast-Mounted  Sensor 

Yes 

- 

- 

- 

25% 

- 

0% 

- 

- 

- 

- 

No 

- 

- 

- 

75% 

- 

100% 

- 

- 

- 

- 

Target  Designation 

Yes 

0% 

0% 

0% 

0% 

- 

100% 

0% 

- 

- 

0% 

No 

100% 

100% 

100% 

100% 

- 

0% 

100% 

- 

- 

100% 

FTL  Targeting 

Yes 

0% 

0% 

0% 

0% 

0% 

0% 

0% 

0% 

0% 

0% 

No 

100% 

100% 

100% 

100% 

100% 

100% 

100% 

100% 

100% 

100% 

Weapon  Stabilization 

Yes 

100% 

25% 

25% 

25% 

25% 

25% 

25% 

0% 

0% 

100% 

No 

0% 

75% 

75% 

75% 

75% 

75% 

75% 

100% 

100% 

0% 

The  ranges  for  attributes  with  qualitative  scales  are  generally  constructed  with 
well-defined  criteria  for  the  different  levels  that  can  be  obtained.  There  are  six  qualitative 
attributes  included  in  the  model:  CBRN  Operation,  Information  System  Growth,  Missile 
Capacity,  Soft  Soil  Mobility,  Static  Stability  Factor,  and  Air  Transportation.  There  are 
also  two  qualitative  attributes — Occupant  Protection  and  Survivability — that  have  seven 
“additive”  categories  each,  which  are  discussed  later  in  this  section. 


A  qualitative  scale  is  characterized  by  using  a  scale  that  is  constructed  with 
discrete  scale  increments,  or  nonoverlapping  bins,  and  with  well-defined  criteria 
associated  with  each  of  the  bins  or  scale  increments.  The  number  of  bins  for  the  six 
qualitative  attributes  range  from  3  to  10.  For  example,  the  CBRN  scale  has  three  bins 
with  the  associated  criteria:  (1)  None;  (2)  Mask  with  threat-specific  filter  system;  and  (3) 
Contains  over  pressure  systems  to  protect  against  CBRN  threats.  For  the  CBRN  attribute 
in  the  HBCT  role,  86%  of  the  alternatives  score  in  bin  2  and  14%  in  bin  3.  No 
alternatives  scored  in  bin  1.  Figure  7  illustrates  the  alternative  histogram  for  the 
HBCT  role. 
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HBCT/  CBRN  Operation 


None  Mask  with  threat-  Contains  overpressure 

specific  filter  system  systems  to  protect 
against  CBRN  threats 


Figure  7.  FIBCT  data  for  CBRN  Operation  fall  within  two  bins:  Mask  with  threat- 
specific  filter  system  (86%)  and  Contains  over  pressure  systems  to  protect  against 

CBRN  threats  (14%). 

When  we  evaluate  CBRN  Operation  for  the  SBCT  role,  we  observe  that  all 
alternative  scores  are  in  bin  2  (Mask  with  threat-specific  filter  system).  See  Appendix  D 
for  table  of  percentage  alternative  scores  by  bin. 

The  last  two  qualitative  attributes  (Occupant  Protection  and  Survivability)  have 
constructed  scales  or  bins,  referred  to  as  bins  A  through  D.  The  attributes  are  each 
comprised  of  seven  categories  (Small  Arms,  RPG,  ATGM,  Tank  KE,  Top  Attack, 
Undervehicle,  and  IED).  Some  of  the  categories  include  multiple  levels,  as  shown  in 
Table  8. 
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Table  8. 


Number  of  levels  used  for  each  Occupant  Protection  and  System 
Survivability  Category. 


Category 

#  Levels 

Occupant  Protection  against  Small  Arms 

6 

Occupant  Protection  against  RPGs 

3 

Occupant  Protection  against  ATGMs 

3 

Occupant  Protection  against  Tank  KE 

1 

Occupant  Protection  against  Top  Attack 

3 

Occupant  Protection  against  Undervehicle 

4 

Occupant  Protection  against  IEDs 

3 

System  Survivability  against  Small  Arms 

6 

System  Survivability  against  RPGs 

3 

System  Survivability  against  ATGMs 

3 

System  Survivability  against  Tank  KE 

1 

System  Survivability  against  Top  Attack 

3 

System  Survivability  against  Undervehicle 

4 

System  Survivability  against  IEDs 

3 

For  Occupant  Protection  against  Small  Arms,  the  HBCT  MtrCr  role  data  includes 
16  data  points  within  bin  A,  2  bin  B,  5  bin  C,  and  13  bin  D.  However,  in  order  to  prevent 
categories  with  more  levels  from  dominating  the  results,  the  data  are  normalized  by 
dividing  by  the  number  of  vehicle  alternatives  for  a  given  role;  in  this  case,  six  vehicles 
are  considered  for  the  MtrCr  role.  This  normalization  allows  for  comparison  across  roles. 
Figure  8  shows  the  normalized  count  for  the  MtrCr  role.  Data  in  bins  A  through  D  sum  to 
six  because  there  are  six  levels  for  the  Occupant  Protection  Against  Small  Arms  category. 


Occupant  Protection  Against  Small  Arms 
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Figure  8.  MtrCr  data  for  Occupant  Protection  Against  Small  Arms,  showing  that  the 

majority  of  the  data  fall  within  bins  A  and  D. 
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Figure  9  illustrates  the  breakdown  of  the  data  across  all  vehicles  in  the  FIBCT  role 
for  the  Occupant  Protection  Against  Small  Arms  category.  The  histogram  graphic 
illustrates  that  the  majority  of  the  scores  fall  in  bins  A  and  D. 
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Figure  9.  A  breakdown  of  Occupant  Protection  Against  Small  Arms  for  HBCT  roles 

showing  that  the  majority  of  the  data  fall  in  bins  A  and  D. 

4.4. 2.1  Performance  Model  Summary 

For  future  versions  of  the  CP  AT  perfonnance  model,  we  suggest  several 

improvements: 

1.  Conduct  further  analysis  of  the  value  function  sensitivity  and  relevancy 
analysis  in  order  to  identify  attributes  that  can  be  eliminated.  Currently, 
the  model  contains  many  attributes  that  do  not  measure  anything; 
meaning,  the  attribute  is  only  relevant  for  one  to  three  alternatives  or  the 
data  for  the  attribute  is  the  same  for  all  alternatives.  A  reduction  in  the 
number  of  attributes  will  allow  for  greater  ease  of  data  collection,  higher 
quality  of  value  function  and  weight  assessments,  and  facilitate  design  and 
weighting  of  a  global  performance  model. 

2.  Simplify  the  protection  and  lethality  attributes  by  developing  only  the 
relevant  categories  that  influence  results.  Weight  the  categories  locally  for 
these  two  attributes,  not  globally  against  all  of  the  other  attributes  in  the 
model.  This  change  will  greatly  simplify  global  weighting  assessments 
and  provide  a  much  more  transparent  and  defendable  value  increment 
assessment  of  the  two  value  functions,  which  is  not  done  in  the  current 
model.  This  change  will  also  enforce  the  integrity  of  the  value  increment. 

3.  Weight  all  attributes  across  all  roles  globally.  Currently,  this  is  not  done 
and  results  in  20  different  models,  one  corresponding  to  each  role.  Due  to 
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this  current  limitation,  alternative  values  from  one  model  should  not  be 
directly  compared  to  those  of  another  model. 


In  general,  the  performance  model  provides  robust  results.  This  is 
primarily  due  to  the  partitioned  structure  of  the  model.  In  other  words,  a  small  number  of 
alternatives  are  considered  for  a  given  role,  so  changes  in  weights  or  the  shape  of  the 
value  function  only  affect  a  small  number  of  alternatives  in  a  given  role.  Given  that  each 
of  the  20  role  models  is  independent,  changes  in  one  model  do  not  affect  the  alternative 
ranking  in  another.  This  may  not  be  the  case  when  all  alternatives  are  considered  in  the 
optimization  and  are  evaluated  in  the  next  chapter  of  this  report. 
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5. 


OPTIMIZATION  REVIEW 


5.1  REVIEW  OF  OPTIMIZATION  FORMULATION 

The  optimization  model  presented  in  the  Fleet  Management  Planning  Decision 
Analysis:  A  Mixed  Integer  Linear  Program  Formulation  working  paper  by  Andrade,  Kao, 
Lawton,  Melander,  and  Rice  (2011)  is  an  integer  linear  program  that  prescribes  yearly 
procurement  and  upgrades  47  vehicles,  across  20  roles.  Not  all  vehicles  are  considered 
for  all  roles  (see  Section  1.2).  There  are  several  possible  objective  functions  (maximize 
overall  fleet  performance,  maximize  final  fleet  perfonnance,  minimize  Operations  and 
Support  (O&S)  cost,  and  minimize  total  cost)  used  in  different  combinations.  The  impact 
of  using  these  different  objective  functions  was  reported  by  Sandia  (Sandia  National 
Laboratory,  2011).  However,  only  one  of  the  objective  functions,  maximize  overall  fleet 
performance,  has  been  used  to  obtain  results  for  senior  leadership  review.  Overall  fleet 
performance  is  defined  by  the  total  value  of  the  all  vehicles  in  service  over  all  periods  of 
the  time  horizon,  subject  to  a  minimum  number  of  brigades  filled  by  the  last  time  period 
(constraints  1.38  and  1.39)  (Andrade  et  ah,  2011).  Other  constraints  limit  yearly 
upgrades,  procurement,  and  budgets. 

Overall,  this  is  a  good  application  for  optimization.  Several  examples  of  similar 
optimization  models  can  be  found  in  the  military  operations  research  literature  (see,  for 
example,  Brown,  Dell,  &  Newman,  2004).  The  following  section  enumerates 
observations  from  our  review  of  the  formulation  from  Andrade  et  al.  (2011). 

1.  Several  parameters  could  be  made  more  general,  as  in  other  similar 
applications  (Brown  et  al.,  2004).  For  example, 

(i)  MissionReqm  could  vary  by  year.  As  written,  the  value  must 
remain  the  same  for  every  year  of  the  planning  horizon.  Clearly, 
the  need  for  vehicles  in  a  specific  mission  area  could  increase  or 
decrease  over  time  and  allowing  this  parameter  to  vary  by  year 
would  be  a  simple  fix. 

(ii)  MaxBDEm  could  also  vary  by  year.  Again,  allowing  this 
parameter  to  vary  by  year  would  be  a  simple  way  to  permit 
changes  over  time  and  could  better  account  for  the  ARFOGEN 
cycle. 

2.  The  penalty  should  not  be  the  same  for  all  deviations  (elastic  variables)  as 
written  in  Equation  (1.10)  of  the  Andrade  et  al.  (2011)  report,  and  should 
potentially  vary  by  year.  As  we  understand  it,  the  elastic  variables  were 
not  implemented  for  the  results  reported  for  senior  leadership  review.  That 
said,  they  are  available  in  the  implementation  (shown  below)  and  this 
implementation  does  not  match  the  documentation. 
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dexpr  float  objFunValNew  = 
sum(r  in  setSupportedRoles,  t  in  setTimePeriodlD) 

Alphafr]  *  NumVehInService[r][t] 

1000000000000.  l*sum(tx  in  setSupportedTransitions,  t  in  setTimePeriodlD) 
iExcessT  ransition[tx]  [t] 

-  1 000000.  l*sum(t  in  setTimePeriodID)(fExcessbudget[t]) 

-  100000000.  l*sum(t  in  setTimePeriodlD)(fExcessOSbudget[t]) 

-  1 000000.  l*sum(t  in  setTimePeriodID)(fExcessRDTEbudget[t]) 

-  1 000000.  l*sum(t  in  setTimePeriodID)(fExcessTotalbudget[t]); 

maximize  objFunValNew; 

From  the  implementation,  we  see  the  penalty  values  are  not  parameter 
values  that  can  be  changed  outside  of  the  implementation.  Furthennore, 
the  values  themselves  appear  to  be  somewhat  arbitrary  and  unrelated  to 
other  objective  function  coefficients.  The  orders  of  magnitude  differences 
used  for  the  penalties  above  together  with  numerical  precision  and  any 
allowed  optimality  gaps  could  make  some  of  these  terms  meaningless. 

Allowing  these  penalty  parameters  to  vary  by  year  will  help  solution  time 
and  reinforce  the  importance  of  near-term  decisions  relative  to  decisions 
closer  to  the  end  of  the  horizon  (see  Brown  et  ah,  2004,  for  more  details). 

The  requirement  constraint  (Equation  [1.11];  in  Andrade  et  al.  [2011])  has 
several  issues  as  written. 

Vm,t  ^  NumInServicej  mt+  ^  ExcessTransitionjimt 

i  e  VehSupported  (m)  j&VehSupported(m),t*<t 


i)  Excess  Transition  j  is  written  as  Excess  Transitions  . ;  mt 


elsewhere  in  the  document. 

ii)  As  written,  the  same  penalty  per  unit  exists  for  being  over  and 
under  the  mission  requirement  level.  Being  able  to  satisfy  mission 
requirements,  but  being  over  is  arguably  far  better  than  being 
under  or  not  being  able  to  satisfy  mission  requirements.  Put  as  a 
simple  question,  would  you  rather  be  10  vehicles  short  or  10 
vehicles  over?  The  above  fonnulation  assumes  there  is 
no  difference. 

iii)  Summing  t*<t  implies  the  variable  should  be  indexed  by  t*  . 
Summing  over  t* ,  however,  implies  that  the  measured  deviation 
from  the  desired  mission  requirement  in  any  year  is  included  as 
part  of  the  measured  deviation  in  all  subsequent  years.  Clearly,  this 
is  not  desired.  One  would  certainly  prefer  being  short  just  one  year 
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and  not  in  subsequent  years.  As  written,  there  is  no  incentive  for 
eliminating  shortages  in  future  years. 


iv) 

I 


As  written,  the  index  (/)  is  not  controlled  in  the  expressions.  The 
value  must  be  summed  again  in  the  expressions. 

ExcessTransition . .  -  V  ExcessTransition..  ,  . 


j  eVehSupported  ( m),t*<t 


j  eTransition(i  ,m),t*<t 


3.  In  Andrade  et  al.  (2011),  Equation  (1.14)  has  many  of  the  same  issues  as 
previously  described  in  Equation  (1.1 1).  As  implemented,  all  of  the  Excess 
Transition  variables  are  set  to  zero  as  the  default  configuration  and  this  is 
the  only  way  the  constraints  will  work  properly.  Clearly,  this  should  be 
revised.  It  is  too  easy  to  be  given  a  goal  that  would  cause  infeasibility  and 
without  the  use  of  this  variable  it  is  impossible  to  tell  how  much  the  goal 
needs  to  be  reduced  without  performing  additional  model  runs. 

4.  In  Andrade  et  al.  (2011),  Equation  (1.27)  is  a  complicating  constraint  that 
can  make  this  model  more  difficult  to  solve.  A  reformulation  should  be 
considered  to  detennine  brigade  lot  sizing  or  change  the  data  so  that  the 
decisions  are  aggregated  at  the  brigade-set  level. 

5.  It  is  assumed  that  what  is  produced  must  be  fielded  in  the  current  period 
plus  some  delay  (usually  two  years).  This  is  a  very  restrictive  limitation 
and  may  not  provide  the  best  equipping  policy  because  money  may  be  left 
on  the  table.  Whole  brigade  lots  cannot  be  built  with  the  remaining  funds 
without  carrying  inventory  into  the  next  year. 

6.  End  effects  are  a  potential  issue.  Maximizing  the  value  of  the  fleet  at  the 
end  does  not  guarantee  anything  about  the  condition  of  the  fleet  one  year 
afterwards.  For  example,  there  could  be  too  many  required  vehicle 
retirements  one  year  after  the  end  of  the  model  horizon.  Model  excursions 
should  extend  the  horizon  to  ensure  enough  years  are  included  so  that 
prescriptions  in  the  initial  years  are  not  unnecessarily  impacted.  This  is 
likely  the  case  because  vehicle  options  offered  late  in  the  time  horizon  are 
never  chosen  by  CP  AT. 

5.2  CPAT  IMPLEMENTATION  REVIEW 

The  purpose  of  the  implementation  review  is  to  ensure  that  the  fonnulation  is 
correctly  implemented  and  that  the  assumptions  as  stated  are  explicitly  modeled.  A 
second  important  consideration  of  this  section  is  to  determine  which  model  parameters 
have  the  greatest  influence  on  CPAT  solutions.  To  conduct  a  review  of  the  CPAT  model 
we  use  the  data  base,  Unrestricted_4.0B_b30.cpat.  This  is  the  same  data  used  for  the 
evaluation  of  the  value  model.  Due  to  time  constraints  for  conducting  the  review,  we  use 
a  reduced  data  set.  Our  analysis  only  includes  excursions  for  the  MBT,  IFV,  and  the  CAV 
vehicle  variants.  The  MBT  and  IFV  roles  were  selected  because  they  contain  the  GCV 
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and  AMOD  variants,  which  are  important  to  any  analysis.  The  CAV  role  is  a  lesser  role, 
but  still  a  driver  in  many  solutions.  Where  a  typical  excursion  using  the  unrestricted  data 
set  may  take  over  an  hour  to  get  a  solution  within  a  0.1%  gap,  using  the  reduced  data  set 
we  are  able  to  run  numerous  excursions  to  a  zero  gap,  with  each  excursion  taking  only  10 
to  15  seconds.  Because  we  use  a  reduced  data  set  for  our  analysis,  conclusions 
concerning  particular  weapon  systems  should  be  avoided. 

We  make  several  sets  of  excursions  in  our  analysis.  The  first  we  refer  to  as  the 
baseline  set.  For  these  excursions,  we  ran  six  funding  profiles:  $1.0B,  $1.25B,  $1.5B, 
$1.75B,  $2. OB,  and  $2.6B.  Our  intent  for  running  these  excursions  is  two-fold.  First,  to 
determine  where  reduced  budget  levels  forced  significant  changes  in  alternative  selection. 
The  second  is  to  establish  a  baseline  for  further  excursions  by  changing  other  parameters 
at  a  given  funding  profile  of  interest.  Appendix  F  contains  the  pivot  tables  showing  the 
selected  alternatives  and  both  the  modernization  levels  and  fielding  over  the  28-year 
horizon  for  these  six  excursions.  Figure  10  summarizes  the  six  baseline  excursions  and 
we  note  several  interesting  patterns. 
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Figure  10.  Baseline  summary  of  six  funding  profiles  shown  in  the  columns 
($Billions)  for  three  roles:  MBT,  IFV,  and  CAV.  The  alternatives  in  bold  are 
being  modernized  and  nonbolded  alternatives  are  being  fielded. 

The  grand  total  of  vehicles  fielded  increases  to  6,904  as  the  budget  increases  until 
$2. 0B;  then  there  is  a  dramatic  drop  to  4,574  vehicles.  This  drop  is  due  to  two  issues. 
First,  is  the  purchase  of  the  GCV.  A  second,  and  more  subtle,  reason  is  that  the  M1A2 
modernization  is  squeezed  out  by  the  GCV  purchase.  We  discuss  the  second  issue  later  in 
this  section.  The  most  important  take  away  from  these  baseline  excursions  is  that  there 
are  interactions  across  roles.  Unlike  what  we  observed  in  the  analysis  of  the  value  model, 
where  alternatives  only  compete  amongst  each  other  within  a  role,  in  the  optimization 
model  the  alternatives  compete  across  roles.  The  optimization  will  ensure  that  each  role  is 
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properly  fielded,  but  the  value-funding  relationship  may  influence  which  alternatives  are 
selected  within  a  role. 

The  five  excursions  based  on  the  $2. OB  funding  profile  are  used  to  determine  how 
an  increase  in  value  for  a  given  alternative  in  one  role  may  affect  the  alternatives  in 
another  role.  The  first  three  excursions  examine  what  would  happen  if  the  value  for  three 
alternatives  is  increased  by  10%.  The  results  of  these  excursions  are  shown  in 
Appendix  E.  As  expected  from  what  we  saw  from  the  sensitivity  analysis  conducted  by 
Sandia  (Sandia  National  Laboratory,  2011)  there  is  no  change  from  the  baseline  results. 
For  the  fourth  excursion,  we  increase  all  of  the  alternatives  contained  in  the  MBT  role 
(role  7)  by  10%  and  find  that  even  increasing  the  value  of  all  of  the  MBT  alternatives  is 
not  enough  to  push  AMOD  back  into  the  solution.  The  results  for  the  fourth  $2. OB 
funding  profile  excursion  are  shown  in  Figure  11. 
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Figure  1 1 .  Summary  of  $2. OB  funding  baseline  profile  with  and  without  (far  right 

column)  RDTE  costs  being  considered. 

We  note  that  when  RDTE  costs  are  not  considered,  AMOD  is  brought  back  into 
the  portfolio.  This  suggests  that  the  ~$  I  .OB  allocated  for  RDTE  costs  in  the  first  two 
years  has  a  very  influential  impact  on  the  solution.  A  similar  analysis  at  the  $1.5B 
funding  profile  shown  in  Figure  12  suggests  that  the  GCV  alternative  is  only  funded  if 
RDTE  costs  are  not  considered. 
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Figure  12.  Summary  of  $  1 .5B  funding  baseline  profile  with  and  without  (far  right 

column)  RDTE  costs  being  considered. 

The  results  shown  in  Figures  11  and  12  suggest  that  RDTE  costs  do  play  a 
significant  role  in  alternative  selection  when  the  overall  budget  is  constrained.  Therefore, 
additional  sensitivity  analysis  should  be  considered  on  alternative  RDTE  funding  profiles 
for  different  vehicle  alternatives. 

The  next  analysis  again  uses  the  $2. OB  funding  profile.  This  time,  we  examine 
how  the  changes  to  the  upper-bound  of  the  number  of  M2A3  ECP  vehicles  that  may  be 
purchased  in  one  period  affect  the  solution.  CP  AT  currently  assumes  that  there  are  72 
role- 1  type  vehicles  in  a  heavy  brigade  and  that  the  maximum  number  of  M2  A3  ECP 
vehicles  allowed  to  be  purchased  for  M2A3  Operation  Iraqi  Freedom  (OIF) 
modernization  is  set  to  216  in  period  5  (FY2017)  and  360  in  periods  6  and  7  (FYs  2018 
and  2019).  In  other  words,  the  baseline  models  can  upgrade  up  to  three  BCTs  in  the  first 
period  and  up  to  five  BCTs  during  the  second  and  third  periods  with  the  ECP  variant,  but 
must  limit  purchases  of  M2A3ECP  for  the  remaining  21  years  to  two  BCTs  per  year.  We 
are  not  sure  how  realistic  this  requirement  is  from  a  policy  perspective,  given  that 
whatever  is  produced  must  be  fielded  two  years  later  (no  inventory  is  allowed).  However, 
it  is  a  necessary  modeling  limitation  because  of  the  33%  M2  A3  OIF  modernization 
requirement  beginning  in  period  6.  To  examine  the  sensitivity  of  the  purchase  restriction 
we  ran  two  excursions,  with  the  first  restricting  the  purchase  limit  to  144  ECPs  per  year 
and  the  second  allowing  up  to  360  ECPs  per  year.  To  allow  for  this  purchase  restriction, 
we  relax  the  modernization  constraint  to  10%  for  the  first  year  and  20%  for  the 
remaining  years. 

Figure  13  shows  the  result  of  this  analysis.  The  top  table  shows  the  vehicle 
modernization  schedule  over  15  years,  the  middle  table  shows  the  purchase  limited  case 
with  a  vehicle  modernization  schedule  over  18  years,  and  the  bottom  table  shows  the 
unlimited  purchase  case  with  a  vehicle  modernization  schedule  over  19  years. 


32 


Row  L»bcl> 

T 

5  6 

7 

8 

9 

10 

11 

12 

13 

14 

IS 

16  17  18 

19 

20 

Grand  Total 

M2A3  0IF 

216  360 

144 

216 

216 

216 

144  144 

72 

1728 

M2  A3  ECP 

216  360 

72 

144 

144 

144 

1080 

GCV 

72 

72 

72 

72 

144  144 

72 

648 

M2A3  ECP 

72 

144 

144 

144  144  144 

144 

144 

1080 

GCV 

72 

144 

144 

144  144  144 

144 

144 

1080 

M3A3CAVOIF 

102  170 

170 

68 

68 

68 

646 

M3A3CAVECP 

102  170 

170 

68 

68 

68 

646 

M1A2  SEP  TUSK 

140 

140 

140 

140 

140 

70 

70 

70 

70 

70 

70 

1120 

Ml  ECP 

140 

140 

140 

140 

140 

70 

70 

70 

70 

70 

70 

1120 

f 

I 

s 

6  7 

8 

9 

10 

11 

12 

13  14 

IS 

16 

17 

18  19  20  21  22 

23 

Grand  Total 

M2A3  0IF 

144 

144  144 

216 

216 

216 

216  216 

116 

1728 

M2A3  ECP 

144 

144  72 

144 

144 

144 

144  144 

144 

1224 

GCV 

72 

72 

72 

72 

72 

72 

72 

SO* 

M2A3ECP 

72 

72 

144 

144 

144  144  144  144  144  72 

1224 

GCV 

72 

72 

144 

144 

144  144  144  144  144  72 

1224 

-  M3A3  CAV  Oif 

102 

102  102 

68 

34 

68 

68 

68 

68  68 

68 

816 

M3A3CAVECP 

102 

102  102 

68 

34 

68 

68 

68 

68  68 

68 

816 

•  MIA 2  SEP  TUSK 

140  140 

140 

140 

140 

140  140 

140  140 

140  70 

70 

70 

1610 

Ml  ECP 

140  140 

140 

70 

70 

70 

70 

70  70 

70 

70 

70 

70 

1120 

AMOD 

70 

70 

70 

70 

70  70 

70 

*90 

Row  Labols  T 

S 

6  7 

8 

9 

10 

11 

12 

13  14 

IS  16  17  18  19  20 

21 

22 

Grand  Total 

M2A3  OIF 

288  288  360 

216 

216 

216 

144 

1728 

M2  A3  ECP 

288  216  288 

144 

144 

144 

72 

1296 

GCV 

72 

72 

72 

72 

72 

72 

432 

M2A3  ECP 

72  72  144  144  144  144  144  144 

144 

144 

1296 

GCV 

72  72  144  144  144  144  144  J44 

144 

144 

1296 

-  M3A3CAVOIF 

102 

170  170  68 

34 

68 

68 

68 

68 

816 

M3A3CAVECP 

102 

170  170  68 

34 

68 

68 

68 

68 

816 

-  MIA2  SEP  TUSK 

70 

140  70 

140 

140 

140 

140 

140 

140  140 

70  70  70  70 

1S40 

Ml  ECP 

70 

140  70 

140 

70 

70 

70 

70 

o 

o 

r* 

70  70  70  70 

1120 

AMOD 

70 

70 

70 

70 

70  70 

420 

Figure  13.  Vehicle  fielding  and  modernization  with  a  $2. OB  funding  profile.  The 
tables  show  the  equipping  profiles  over  time.  The  top  table  is  the  baseline  profile, 
the  middle  table  shows  the  case  where  the  M2A3  ECP  fielding  is  limited  to  two 
BDEs  for  the  first  three  periods,  and  the  bottom  table  removes  the  BDE  limit  on 
M2A3  ECP  fielding.  The  vehicles  being  modernized  are  shown  in  bold  and  the 
number  of  each  vehicle  being  fielded  are  shown  in  the  columns  under  their 
respective  bold  modernization  entries  for  each  period.  For  example,  in  the  top 
table  under  the  eighth  period  (FY2020)  there  are  144  M2A3  ECPs  and  72  GCVs 
being  fielded,  which  will  replace  216  M2  A3  OIF  vehicles. 

There  are  several  interesting  things  to  note  about  this  analysis  and  it  also 
demonstrates  a  fundamental  tenant  of  optimization:  when  we  apply  a  constraint  to  a 
problem,  we  cannot  expect  a  better  outcome.  For  example,  even  with  the  relaxed 
modernization  constraint,  the  restricted  purchases  case  finishes  M2A3  OIF  modernization 
the  same  year  as  the  baseline  case,  resulting  in  a  0.3%  increase  in  overall  portfolio 
performance  and  an  increase  of  vehicle  modernization  of  16%  over  the  baseline  case. 
Additionally,  the  same  number  of  GCVs  is  purchased  in  the  restricted  case  as  in  the 
baseline;  however,  AMOD  is  also  purchased  in  the  restricted  case.  For  the  unrestricted 
purchases  case,  modernization  is  accomplished  a  year  sooner,  resulting  in  an  increase  of 
14%  in  overall  modernization  and  AMOD  is  still  purchased  with  an  overall  performance 
bump  over  the  baseline  case  of  0.6%.  Based  on  these  findings,  we  suggest  more  studying 
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of  the  maximum  purchases  allowed  and  relaxation  of  the  modernization  requirements 
using  elastic  constraints  to  allow  exploration  of  more  efficient  modernization  policies  that 
are  not  based  solely  on  a  fixed  schedule. 

5.3  OTHER  ISSUES 

Overall,  the  GUI  allows  CPAT  to  be  used  by  novice  users  unaccustomed  to 
running  optimization  models.  Except  for  the  penalties  on  elastic  variables,  the  user  can 
access  most  of  the  input  parameters  of  the  optimization  model.  The  developers  put  much 
of  their  effort  into  generation  of  output,  which  makes  sense  given  that  the  ultimate  user  is 
the  PM  office.  We  did  not  have  access  to  the  underlying  code  of  the  output  module  and 
verification  of  the  output  was  not  our  focus.  However,  through  our  analysis,  we  did  find  a 
couple  of  minor  issues  that  should  be  addressed. 

To  illustrate  our  first  issue  with  CPAT  output,  we  use  the  “Mission  Fielding 
Schedule”  from  the  CPAT  Results  Explorer  reproduced  in  Figure  14. 
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Figure  14.  Vehicle  fielding  for  the  $2. OB  funding  profile  from  the  CPAT 

Results  Explorer. 

Correctness  of  this  chart  is  not  the  issue  and  it  can  be  verified  to  contain  identical 
quantities,  as  shown  the  $2. OB  baseline  funding  profile  case  in  Figure  13.  What  is  not 
apparent  from  this  chart,  but  can  be  seen  in  Figure  12,  is  that  GCV  fielding  begins  for 
modernization  of  M2A3  ECP  only  three  years  after  the  purchase  of  the  last  of  1,080  ECP 
variants  that  were  used  to  modernize  the  M2  A3  OIF.  This  is  not  apparent  in  Figure  14 
because  fielding  and  modernization  results  are  separated  in  the  CPAT  Results  Explorer. 
This  policy  behavior  is  not  likely  something  leadership  would  support  and  could  be  easily 
addressed  with  additional  CPAT  constraints. 

The  other  issue  concerns  the  correctness  of  the  first  several  time  periods  of  the 
Acquisition  Expenditures  from  the  “Total  Expenditure  vs.  Budget”  page  from  the  Results 
Explorer.  Figure  15  shows  this  table  for  the  $1.25B  funding  level  and  the  far  right 
column  shows  our  calculation.  Note  the  inconsistencies  for  FY2015  and  FY2016.  Similar 
results  are  found  in  the  other  cases  for  the  first  few  periods. 


34 


FYI3 

nru 

ms 

FY16 

FY17 

mu 

*Y19 

FY20 

mt 
fY22 
rv 23 

FY7-4 

FYJS 

fY26 

f-Y2; 

FY28 

rY29 

mo 

FY31 

FY3? 

fY33 

f-YJ4 

f-Y3i 

mo 

m7 

tym 

ms 

FY40 


Figure  15.  Acquisition  Expenditure  deficiencies  for  $1.25B  funding  profile. 

As  mentioned  earlier,  CPAT  fields  vehicles  as  brigade  sets.  The  assumption  is 
that  what  is  produced  must  be  fielded  at  a  set  interval,  in  most  cases  two  years  after 
production.  Partial  sets  are  not  allowed;  in  other  words,  inventory  is  not  carried  over  from 
one  period  to  another.  One  of  the  obvious  drawbacks  to  this  approach  is  that  all  of  the 
allowable  budget  appropriation  may  not  be  used,  as  shown  in  Figure  16.  The  last  column 
shows  the  budget  surplus  for  the  constant  $2. OB  funding  profile. 


Time  Period 
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RDTE  Budget 

Total  Budget 

Difference 

FY13 

$2,000,000,000 

$0 

$1,431,400,000 

$1,431,400,000 

$568,600,000 

FY14 

$2,000,000,000 

$0 

$1,567,900,000 

$1,567,900,000 

$432,100,000 

FY15 

$2,000,000,000 

$340,578,000 

$1,576,100,000 

$1,916,678,000 

$83,322,000 

FY16 

$2,000,000,000 

$1,001,630,000 

$970,700,000 

$1,972,330,000 

$27,670,000 

FY17 

$2,000,000,000 

$1,580,942,000 

$407,600,000 

$1,988,542,000 

$11,458,000 

FY18 

$2,000,000,000 

$1,548,812,000 

$55,100,000 

$1,603,912,000 

$396,088,000 

FY19 

$2,000,000,000 

$1,548,812,000 

$76,000,000 

$1,624,812,000 

$375,188,000 

FY20 

$2,000,000,000 

$1,548,812,000 

$42,200,000 

$1,591,012,000 

$408,988,000 

FY21 

$2,000,000,000 

$1,992,520,000 

$2,500,000 

$1,995,020,000 

$4,980,000 

FY22 

$2,000,000,000 

$1,992,520,000 

$0 

$1,992,520,000 

$7,480,000 

FY23 

$2,000,000,000 

$1,992,520,000 

$0 

$1,992,520,000 

$7,480,000 

FY24 

$2,000,000,000 

$1,992,520,000 

$0 

$1,992,520,000 

$7,480,000 

FY25 
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$1,992,520,000 

$7,480,000 
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$1,992,520,000 

$7,480,000 
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$224,480,000 
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$2,000,000,000 

FY32 

$2,000,000,000 

$0 

$0 

$0 

$2,000,000,000 

FY33 
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$2,000,000,000 

$0 

$0 

$0 
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$0 

$2,000,000,000 

FY39 

$2,000,000,000 

$0 

$0 

$0 

$2,000,000,000 

FY40 

$2,000,000,000 

$0 

$0 

$0 

$2,000,000,000 

Figure  16.  Total  Budget  versus  the  $2. OB  funding  profile  budget. 
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Additionally,  fielding  decisions  currently  show  high  volatility  from  one  period  to 
the  next.  This  behavior  is  undesirable  because  most  manufacturers  are  not  going  to  vary 
production  by  50%  or  more  from  one  year  to  the  next  without  considerable  expense  to  the 
government.  The  production  volatility  may  be  reduced  by  using  inventory  to  smooth  out 
the  deviations  from  year  to  year. 
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6. 


CONCLUSIONS  AND  RECOMMENDATIONS 


CP  AT  is  a  good  application  for  optimization.  At  its  core,  CP  AT  is  an  optimization 
model;  however,  the  CPAT  objective  function  is  derived  using  multiattribute  decision 
analysis  techniques.  Therefore,  we  approached  this  evaluation  by  first  focusing  on  the 
verification  and  validation  of  the  performance  model  that  produces  the  CPAT  objective 
function  coefficients.  We  then  verified  the  CPAT  optimization  fonnulation  and  ensured 
that  the  OPL  code  was  implemented  as  indicated  by  the  fonnulation.  The  last  step  was  to 
validate  the  overall  CPAT  model  by  observing  changes  to  CPAT  solutions  when  we 
varied  in  the  most  sensitive  model  parameters.  We  would  like  to  highlight  the  following 
results  of  this  analysis: 

1.  In  general,  the  performance  model  provides  robust  results.  This  is 
primarily  because  only  a  small  number  of  alternatives  are  considered  for  a 
given  role.  Changes  in  weights  or  the  shape  of  the  value  function  only 
affect  a  small  number  of  alternatives  in  a  given  role. 

2.  The  model  contains  many  attributes  that  are  not  very  relevant  and 
essentially  do  not  contribute  to  the  differentiation  among  the  alternatives. 

3.  It  is  possible  to  simplify  the  protection  and  lethality  subobjectives  by 
utilizing  only  the  relevant  categories  that  influence  results. 

4.  All  attributes  should  be  weighted  globally  across  all  roles.  The  current 
weighting  system  results  in  20  different  models,  one  corresponding  to 
each  role.  As  a  result  of  this  current  limitation,  alternative  values  from  one 
model  should  not  be  directly  compared  to  values  of  another  model. 

5.  Several  parameters  could  be  more  general  in  the  fonnulation.  For 
example,  MaxBDEm  and  MissionReqm  could  vary  by  year. 

6.  In  Equation  (1.10)  from  Andrade  et  al.  (2011),  the  penalty  should  not  be 
the  same  for  all  deviations  (elastic  variables);  it  should  potentially  vary  by 
year. 

7.  Several  constraints  (Equations  [1.11]  and  [1.14];  Andrade  et  al.  [2011]) 
have  issues  as  they  are  currently  written.  For  example,  the  same  penalty 
per  unit  exists  for  being  over  and  under  the  mission  requirement  level. 

8.  In  Andrade  et  al.  (2011),  Equation  (1.27)  is  a  potentially  unnecessary 
complicating  constraint  that  increases  solution  time.  Consider  a  different 
fonnulation  to  determine  brigade  lot  sizing  or  change  the  data  so  that  the 
decisions  are  aggregated  at  the  brigade-set  level. 

9.  It  is  implied  that  what  is  produced  must  be  fielded  as  a  brigade  set;  in  most 
cases,  two  years  after  production.  Due  to  this  constraint,  all  of  the 
allowable  budget  appropriation  may  not  be  used. 

10.  End  effects  are  a  potential  issue.  Maximizing  the  value  of  the  fleet  at  the 
end  does  not  guarantee  anything  about  the  condition  of  the  fleet  one  year 
afterwards.  For  example,  there  could  be  too  many  required  vehicle 
retirements  one  year  after  the  end  of  the  model  horizon.  Model  excursions 
should  extend  the  horizon  to  ensure  that  enough  years  are  included  so  that 
prescriptions  in  the  initial  years  are  not  unnecessarily  impacted.  This  is 
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likely  the  case  because  vehicle  options  offered  late  in  the  time  horizon  are 
never  chosen  by  CP  AT. 

1 1 .  Unlike  the  value  model,  where  alternatives  only  compete  amongst  each 
other  within  a  role,  in  the  optimization  model  alternatives  compete  across 
roles.  Therefore,  value  trade-offs  are  critical  in  a  reduced  budget 
environment. 

12.  CP  AT  results  are  very  sensitive  to  RDTE  funding  profiles  for  different 
vehicle  alternatives. 

13.  CP  AT  results  are  sensitive  to  maximum  purchases  allowed  parameters 
found  on  the  fielding  schedule.  Elastic  constraints  should  be  considered  to 
allow  exploration  of  more  efficient  modernization  policies  that  are  not 
based  solely  on  a  fixed  schedule. 

Based  on  our  technical  review  of  the  CPAT  model,  we  have  several  near-  and 
long-term  recommendations.  The  near-tenn  recommendations  focus  on  issues  that  can  be 
addressed  with  the  current  model  and  data.  The  long-term  recommendations  focus  on 
larger  structural  changes  and  extensions  of  the  current  model. 

6.1  Near-Term  Recommendations 

1.  Study  the  effects  of  removing  ineffective  value  measures.  We  suggest 
beginning  with  those  measures  that  do  not  use  a  full  range  of  the  scale  for 
a  continuous  measure  (highlighted  in  Section  4.4)  as  well  as  the  qualitative 
and  binary  measures  that  do  not  show  much  variation  in  scale  (or  value). 
See  Appendix  C  for  details  on  value  measure  range  analysis. 

2.  Aggregate  the  level  of  detail  at  the  brigade-set  level.  With  the  current 
version  of  the  model,  we  see  no  need  to  run  CPAT  at  the  vehicle  level  of 
detail.  Currently,  all  fielding  and  modernization  decisions  are  being  made 
at  the  brigade-set  level,  which  implies  that  all  production  decisions  must 
also  be  made  at  the  brigade  set  level  with  a  given  delay.  This  aggregation 
will  greatly  simplify  the  model  by  allowing  for  the  removal  of  the 
complicating  constraint  (1.27)  and  will  allow  for  much  faster  solution 
times  and  a  smaller  optimal  gap  for  most  instances.  If  vehicle  level  of 
detail  is  desired,  it  can  be  computed  very  efficiently  in  a  post  processing 
environment. 

6.2  Long-Term  Recommendations 

1.  Detennine  a  “single”  objective  value  model  by  reducing  the  number  of 
attributes  and  reweighting  across  all  roles.  This  will  allow  users  to  make 
direct  value  comparisons  between  roles  and  strengthen  the  additive 
assumption  implied  by  this  mixed-integer  linear  program. 

2.  Fund  a  prototype  CPAT  Version  2  that  will  consider  separate  decisions  for 
production  and  allow  inventory  to  be  carried  from  one  period  to  the  next. 
This  next  generation  CPAT  will  then  decouple  the  fielding  of  BDE  sets 
from  the  vehicle  production  decisions  and  provide  for  cheaper  long-run 
solutions  with  greater  fleet  performance. 


38 


APPENDIX  A.  DESCRIPTIVE  STATISTICS  SUMMARY  TABLE 


This  descriptive  statistics  summary  table  illustrates  the  model  attributes 
considered  for  each  vehicle  role,  the  unit  of  measure  for  each  attribute,  and  the  scales 
used  for  each  vehicle  type.  The  model’s  49  attributes  and  units  of  measure  are  listed  in 
the  rows  and  the  20  vehicle  types  are  listed  in  the  columns  across  the  top  of  the  table.  The 
bottom  of  the  table  lists  the  number  of  attributes  used  by  each  vehicle  role.  The  table 
highlights  in  color  the  scales  used  for  each  attribute  across  the  20  vehicle  roles  (red 
indicates  1  scale  used  for  all  roles,  blue=  scale  1,  green=  scale  2,  purple=  scale  3).  Cells 
highlighted  in  gray  are  not  used  for  the  role  and  cells  with  a  bisecting  red  line  are 
classified.  In  many  cases,  only  one  scale  is  used  (indicated  in  solid  red)  or  there  is  one 
scale  for  HBCT  (indicated  in  blue)  and  a  second  scale  for  SBCT  roles  (indicated 
in  green). 


Legend 

classified  data 

measure  not  used  for  this  role 

range  same  for  all  roles 

scale  1 

scale  2 

scale  3 

Descriptive  Statistics  Summary  Table 

Role-Attribute-Range  Mapping 


Occupant  Protection  against  Sm  Anns  &  Md 
Cal  Q 

Occupant  Protection  against  RPG  Q 

Occupant  Protection  against  ATGM  Q 

Occupant  Protection  against  T ank  KE  Q 

Occupant  Protection  against  T op  Attack  Q 

Occupant  Protection  against  Undervehicle  Q 

Occupant  Protection  against  1ED  C 

CBRN  Operation  C 

Survivability  against  Sm  Arms  &  Med  Cal  Q 
Survivability  against  RPG  Q 

Survivability  against  ATGM  Q 

Survivability  against  Tank  KE  Q 

Survivability  against  Top  Attack  Q 

Survivability  against  Undervehicle 
Survivability  against  IED  Q 

Silent  Watch  L 


Available  Electrical  Power 
Exportable  Electrical  Power 
Information  System  Growth 
Effective  Range  vs.  Tanks 
Effective  Range  vs.  IFVs 
Effective  Range  vs.  Light  Armor 
Effective  Range  vs.  Personnel  (point  t: 
Effective  Range  vs.  Personnel  (area  tai 
Efficiency  vs.  ADA  site 
Efficiency  vs.  artillery  platoon 
Efficiency  vs.  mech  platoon 
Efficiency  vs.  MRL 
Efficiency  vs.  dismounted  crew 
Missile  Capabflity 
ATGM  Fire  Control 
Mast  Mounted  Sensor 
Target  Designation 
FTL  Targeting 
Weapon  Stabilization 
Transition  from  Firing  to  Moving 


Qualitative 
Meters 

Meters 
Meters 

Number  of  Round: 
Number  of  Round: 
Number  of  Round: 
Number  of  Round: 
Number  of  Rounds 
Qualitative 

Binary 

Binary 

Binary 

Binary 

Seconds 


Personnel  Carry 
Litter  Carry 


Soft  Soil  Mobility 
Max  Speed 
Turning  Diameter 
Static  Stability  Factor 
Vehicle  Width 
Dash  Speed 
Air  Transportability 
Cruising  Range 


Operational  Availability  -  Initial 
Operational  Availability  -  Worst  Case 
Sustainment  Availability 


Percentage 
Percentage 
Percentage  | 

Attributes  Used  By  Role 
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APPENDIX  B.  SUMMARY  TABLE  HEAVY  BRIGADE  COMBAT 
TEAM  (HBCT)/STRYKER  BRIGADE  COMBAT  TEAM  (SECT) 

ATTRIBUTE  SCALES 


The  table  illustrates  when  the  possible  range,  defined  as  the  minimum  and 
maximum  on  the  value  function  scale,  is  the  same  for  a  given  attribute  across  all  HBCT 
and  SBCT  roles.  Cells  highlighted  in  green  indicate  that  all  vehicles  in  the  role  use  the 
same  possible  range.  Cells  highlighted  in  yellow  indicate  when  the  majority  of  the 
vehicles  use  the  same  range;  for  example,  the  value  function  for  the  Available  Electrical 
Power  attribute  uses  the  same  x-axis  scale  for  all  vehicles  in  the  HBCT  role,  but  two 
y-axis  scales.  For  27  of  the  49  attributes  in  the  model,  the  same  scale  is  used  across  all  20 
HBCT/SBCT  roles. 


Subobjective 

Attribute 

HBCT 

SBCT 

Occupant  Protection  against  Small  Anns  &  Medium  Caliber 

No-same  x  scale;  2  ys 

Yes 

Occupant  Protection  against  RPG 

No-same  x  scale;  2  ys 

Yes 

Occupant  Protection  against  ATGM 

No- same  x  scale;  2  ys 

Yes 

Occupant  Protection  against  Tank  KE 

No 

Yes 

Occupant  Protection  against  Top  Attack 

No-same  x  scale;  2  ys 

Yes 

Occupant  Protection  against  Undervehiele 

No-same  x  scale;  2  ys 

Yes 

Occupant  Protection  against  IED 

No-same  x  scale;  2  ys 

Yes 

Survivability 

CBRN  Operation 

Yes 

Yes 

Survivability  against  Small  Arms  &  Medium  Caliber 

No- same  x  scale;  2  ys 

Yes 

Survivability  against  RPG 

No-same  x  scale;  2  ys 

Yes 

Survivability  against  ATGM 

No-same  x  scale;  2  ys 

Yes 

Survivability  against  Tank  KE 

No 

Yes 

Survivability  against  Top  Attack 

No-same  x  scale;  2  ys 

Yes 

Survivability  against  Undervehicle 

No 

Yes 

Survivability  against  IED 

No 

Yes 

Silent  Watch 

Yes 

Yes 

Available  Electrical  Power 

No-same  x  scale;  2  ys 

Yes 

Growth 

Exportable  Electrical  Power 

No- SPA  is  different 

Yes 

Information  System  Growth 

No- same  y  scale;  2  possible 
orders  for  xs 

No- same  y  scale,  2  possible 
orders  for  xs 

Effective  Range  vs.  Tanks 

Yes 

No-2  possible  combinations 

Effective  Range  vs.  IFVs 

Yes 

Yes- onfy  ATGM  listed 

Lethality 

Effective  Range  vs.  Light  Armor 

Yes 

No-2  possible  combinations 

Effective  Range  vs.  Personnel  (point  target) 

Yes 

No-2  possible  combinations 

Effective  Range  vs.  Personnel  (area  target) 

No-2  possible  combinations 

No-2  possible  combinations 

Efficiency  vs.  ADA  site 

Yes 

Yes 

Efficiency  vs.  Artillery  platoon 

Yes 

Yes 

Efficiency  vs.  Mech  platoon 

Yes 

Yes 

Efficiency  vs.  MRL 

Yes 

Yes 

Efficiency  vs.  Dismounted  crew 

Yes 

Yes 

Lethality 

Missile  Capability 

Yes 

Yes 

ATGM  Fire  Control 

N/A 

Yes 

Mast-Mounted  Sensor 

Yes 

Yes 

Target  Designation 

Yes 

Yes 

FTL  Targeting 

Yes 

Yes 

Weapon  Stabilization 

Yes 

Yes 

Transition  from  Firing  to  Moving 

Yes- only  SPA  listed 

Not  Applicable 

Personnel  Payload 

Personnel  C’any 

No-2  possible  combinations 

No-2  possible  combinations 

Litter  Carry 

Yes- only  Med  listed 

Yes- only  MEV  listed 

Soft  Soil  Mobility 

Yes 

Yes 

Max  Speed 

Yes 

Yes 

Turning  Diameter 

Yes 

Yes 

Mobility 

Static  Stability  Factor 

Yes 

Yes 

Vehicle  Width 

Yes 

Yes 

Dash  Speed 

Yes 

Yes 

Air  Transportability 

Yes 

Yes 

Cruising  Range 

Yes 

Yes 

Operational  Availability  -  Initial 

Yes 

Yes 

Sustainability 

Operational  Availability  -  Worst  Case 

Yes 

Yes 

Sustainment  Availability 

Yes 

Yes 

Part  Commonality 

Yes 

Yes 

used  in  analysis 

Embedded  Diagnostics 

Yes 

Yes 

Mean  Time  Between  Essential  Function  Failure 

Yes 

Yes 
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APPENDIX  C.  QUANTITATIVE  ATTRIBUTES  -  PERCENT  RANGE  USED  WITHIN  EACH 

VALUE  FUNCTION 


Our  analysis  shows  that  the  actual  range  of  the  data  often  does  not  utilize  the  full  possible  range  of  the  value  functions. 
For  this  analysis,  we  define  the  possible  range  as  the  minimum  and  maximum  on  the  value  function  scale,  and  the  actual  range 
as  the  minimum  and  maximum  values  of  the  data  that  is  input  into  the  model. 

This  table  shows  the  percentage  of  the  actual  range  used  by  the  data  in  the  model.  Cells  highlighted  in  yellow  indicate 
that  0%  of  the  range  was  used  or  no  variability  in  the  data;  meaning  all  vehicle  data  within  the  role  are  the  same.  Red  cells 
indicate  that  <  25%  of  the  possible  range  is  used,  White  26%-49%,  and  green  50%-100%.  Light-gray  cells  have  data  that 
extend  beyond  the  value  function’s  established  range  and  dark  gray  cells  were  not  assessed  for  the  role. 


Percent  Range  Used 

HBCT 

SBCT 

Attribute 

MBT 

I  FA 

CAV 

FIST 

Eng 

SPA 

MtrCr 

C2  (H) 

Med 

GP 

ICV 

ATGM 

C2(S) 

RV 

MCV 

FSV 

ESV 

MEV 

NBCRV 

MGS 

Silent  Watch 

183.33% 

131.25% 

131.25% 

131.25% 

131.25% 

14.58% 

47.92% 

47.92% 

47.92% 

47.92% 

31.25% 

31.25% 

31.25% 

31.25% 

31.25% 

31.25% 

31.25% 

31.25% 

31.25% 

31.25% 

Available  Electrical  Power 

60.60% 

45.11% 

45.11% 

56.87% 

51.14% 

51.47% 

39.71% 

73.53% 

73.53% 

39.71% 

20.59% 

20.59% 

20.59% 

20.59% 

22.06% 

20.59% 

20.59% 

20.59% 

20.59% 

20.59% 

Exportable  Electrical  Power 

0.00% 

48.89% 

0.00% 

0.00% 

0.00% 

0.00% 

0.00% 

48.89% 

0.00% 

0.00% 

0.00% 

0.00% 

0.00% 

0.00% 

0.00% 

0.00% 

0.00% 

0.00% 

0.00% 

0.00% 

Effective  Range  vs.  Tanks 

2.40% 

11.25% 

11.25% 

100.00% 

105.00% 

- 

- 

- 

- 

- 

100.00% 

0.00% 

- 

- 

- 

- 

100.00% 

- 

- 

0.00% 

Effective  Range  vs.  IFVs 

0.00% 

11.25% 

11.25% 

100.00% 

105.00% 

- 

- 

- 

- 

100.00% 

0.00% 

- 

- 

- 

- 

100.00% 

- 

- 

0.00% 

Effective  Range  vs.  Light  Armor 

0.00% 

22.50% 

22.50% 

155.00% 

165.00% 

- 

- 

- 

- 

155.00% 

0.00% 

- 

155.00% 

- 

155.00% 

155.00% 

- 

- 

0.00% 

Effective  Range  vs.  Personnel  (point 
target) 

100.00% 

25.71% 

0.00% 

14.29% 

14.29% 

0.00% 

0.00% 

0.00% 

0.00% 

0.00% 

0.00% 

Effective  Range  vs.  Personnel  (area 
target) 

97.14% 

32.00% 

0.00% 

16.00% 

16.00% 

0.00% 

0.00% 

0.00% 

0.00% 

0.00% 

0.00% 

Efficiency  vs.  ADA  site 

- 

- 

- 

- 

- 

0.00% 

0.00% 

- 

- 

- 

- 

- 

- 

- 

0.00% 

- 

- 

- 

- 

- 

Efficiency  vs.  Artillery  platoon 

- 

- 

- 

- 

- 

0.00% 

0.00% 

- 

- 

- 

- 

- 

- 

- 

0.00% 

- 

- 

- 

- 

- 

Efficiency  vs.  Mech  platoon 

- 

- 

- 

- 

- 

0.00% 

0.00% 

- 

- 

- 

- 

- 

- 

- 

0.00% 

- 

- 

- 

- 

- 

Efficiency  vs.  MRL 

- 

- 

- 

- 

- 

0.00% 

0.00% 

- 

- 

- 

- 

- 

- 

- 

0.00% 

- 

- 

- 

- 

- 

Efficiency  vs.  Dismounted  crew 

- 

- 

- 

- 

- 

0.00% 

0.00% 

- 

- 

- 

- 

- 

- 

- 

0.00% 

- 

- 

- 

- 

- 

Transition  from  Firing  to  Moving 

- 

- 

- 

- 

- 

0.00% 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

Personnel  Carry 

- 

33.33% 

0.00% 

42.86% 

50.00% 

- 

42.86% 

114.29% 

0.00% 

0.00% 

0.00% 

0.00% 

0.00% 

0.00% 

0.00% 

- 

0.00% 

0.00% 

Litter  Carry 

- 

- 

- 

- 

- 

0.00% 

- 

- 

- 

- 

- 

- 

- 

- 

0.00% 

- 

- 

Max  Speed 

0.00% 

2.00% 

2.00% 

54.00% 

54.00% 

2.00% 

54.00% 

58.00% 

58.00% 

54.00% 

8.00% 

8.00% 

8.00% 

8.00% 

8.00% 

8.00% 

8.00% 

8.00% 

8.00% 

8.00% 

Turning  Diameter 

0.00% 

35.00% 

35.00% 

200.00% 

200.00% 

50.00% 

215.00% 

245.00% 

230.00% 

215.00% 

13.33% 

13.33% 

13.33% 

13.33% 

13.33% 

13.33% 

13.33% 

13.33% 

13.33% 

13.33% 

Vehicle  Width 

0.00% 

19.44% 

19.44% 

58.33% 

19.44% 

11.11% 

33.33% 

54.33% 

33.33% 

40.78% 

0.00% 

0.00% 

0.00% 

0.00% 

0.00% 

0.00% 

0.00% 

0.00% 

0.00% 

0.00% 

Dash  Speed 

0.00% 

54.52% 

66.67% 

77.33% 

72.00% 

40.00% 

66.00% 

58.67% 

66.67% 

86.67% 

14.00% 

13.00% 

12.00% 

14.00% 

15.00% 

13.00% 

10.00% 

10.00% 

12.00% 

14.00% 

Cruising  Range 

0.40% 

22.84% 

57.87% 

91.51% 

77.33% 

17.29% 

48.62% 

103.33% 

92.13% 

70.80% 

101.89% 

101.00% 

106.44% 

116.73% 

102.59% 

102.68% 

100.96% 

99.08% 

1 10.60% 

100.22% 

Operational  Availability  -  Initial 

5.76% 

7.76% 

7.87% 

19.22% 

9.31% 

43.54% 

17.16% 

44.71% 

42.27% 

65.56% 

2.14% 

0.93% 

1.52% 

1.51% 

0.39% 

2.02% 

10.39% 

0.61% 

41.96% 

17.26% 

Operational  Availability  -  Worst  Case 

10.44% 

12.33% 

12.91% 

28.89% 

14.60% 

57.58% 

24.86% 

59.38% 

55.82% 

86.89% 

3.59% 

0.79% 

5.40% 

1.05% 

1.26% 

4.50% 

15.08% 

1.85% 

47.10% 

22.42% 

Sustainment  Availability 

6.44% 

8.78% 

9.31% 

17.64% 

16.82% 

46.07% 

15.24% 

48.36% 

44.36% 

68.00% 

2.57% 

1.52% 

2.12% 

1.52% 

0.83% 

3.07% 

11.42% 

0.91% 

44.26% 

18.62% 

KEY: 

(-)  Not  assessed  for 
the  role 

0%  (no  variability) 

<25% 

26%-49% 

50%-100% 

>  100% 
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APPENDIX  D.  QUALITATIVE  ATTRIBUTES-PERCENTAGE 
ALTERNATIVE  SCORES  BY  BIN 


The  model  has  six  qualitative  attributes  that  are  characterized  by  using  a  scale 
constructed  with  discrete  scale  increments  in  nonoverlapping  bins.  The  tables  below 
show  the  percent  alternative  scores  by  bin.  For  example,  in  the  first  table  for  the  CBRN 
Operation  attribute  in  the  HBCT  role,  86%  of  the  alternatives  score  in  bin  2  (Mask  with 
threat-specific  filter  system)  and  14%  in  bin  3  (Contains  over  pressure  systems  to  protect 
against  CBRN  threats),  no  alternatives  score  in  bin  1.  For  the  SBCT  role,  100%  of  the 
alternatives  score  in  bin  2. 


CBRN  Operation 

Percent 

HBCT 

SBCT 

None 

- 

- 

Mask  with  threat-specific  filter 
system 

86% 

100% 

Contains  over  pressure  systems 
to  protect  against  CBRN  threats 

14% 

- 

Information  Systems  Growth 

Percent 

HBCT 

SBCT 

None 

- 

- 

HMS  Suite 

57% 

55% 

GMR  Suite 

17% 

30% 

WIN-T 

- 

- 

Support  Combination  of  Radio 

26% 

15% 

Suites 

Missile  Capability 

Percent 

HBCT 

SBCT 

None 

3% 

75% 

TOW 

14% 

10% 

TOW-RF 

59% 

- 

Fire  and  Forget 

24% 

15% 

Multiple  Fire  and  Forget 

- 

- 

45 


Soft  Soil  Mobility 

Percent 

HBCT 

SBCT 

Bin  1 

4% 

- 

Bin  2 

12% 

40% 

Bin  3 

13% 

60% 

Bin  4 

- 

- 

Bin  5 

- 

- 

Bin  6 

9% 

- 

Bin  7 

7% 

- 

Bin  8 

33% 

- 

Bin  9 

7% 

- 

Bin  10 

14% 

- 

Static  Stability  Factor 

Percent 

HBCT 

SBCT 

Bin  1 

6% 

- 

Bin  2 

81% 

100% 

Bin  3 

9% 

- 

Bin  4 

4% 

- 

Bin  5 

- 

- 

Air  Transportation 

Percent 

HBCT 

SBCT 

C-5 

7% 

- 

C-17 

68% 

50% 

C-130 

25% 

50% 

Legend:  A  dash  (-)  in  the  table  indicates  not  applicable. 
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APPENDIX  E.  PIVOT  TABLES 


The  pivot  tables  below  show  the  excursions  we  ran  for  each  of  the  six  funding 
profiles:  $1.0B,  $1.25B,  $1.5B,  $1.75B,  $2. OB,  and  $2.6B.  Our  intent  for  running  these 
excursions  was:  (1)  to  detennine  where  reduced  budget  levels  forced  significant  changes 
in  alternative  selection;  and  (2)  to  establish  a  baseline  for  further  excursions  by  changing 
other  parameters  at  a  given  funding  profile  of  interest.  Detailed  discussions  of  these 
tables  are  referenced  in  the  body  of  the  report. 


Baseline  excursion  at  $1.0B 


MissionID 

(All) 

Ll 

FundingProfile 

1 

.T 

Change 

Baseline 

.T 

Sum  of  FieldedCount 

Row  Labels 

Column  Labels  [_I_ 
5 

6 

7 

8 

9 

10 

11 

12 

13 

14 

15 

16 

17 

18 

19 

20 

21 

22 

23 

24 

25 

26  Grand  Total 

-  M2A30IF 

2L6 

360 

288 

144 

72 

72 

72 

72 

144 

144 

144 

1728 

M2A3ECP 

216 

360 

288 

144 

72 

72 

72 

72 

144 

144 

144 

1728 

M1ECP 

TO 

70 

70 

70 

70 

70 

70 

70 

70 

70 

700 

AMOD 

70 

70 

70 

70 

70 

70 

70 

70 

70 

70 

700 

M3A3CAV  OIF 

102 

120 

136 

68 

68 

68 

68 

68 

34 

34 

816 

M3A3CAV  ECP 

1D2 

170 

136 

68 

68 

68 

68 

68 

34 

34 

816 

M1A2  SEP  TUSK 

140 

70 

140 

140 

140 

140 

140 

140 

140 

140 

140 

140 

70 

1680 

Ml  ECP 

140 

70 

140 

140 

70 

/O 

/O 

A) 

/O 

70 

/O 

70 

70 

1120 

AMOD 

/O 

A) 

/o 

A) 

/O 

70 

/o 

70 

560 

Baseline  excursion  at  $1.25B 


MissionID 

(All) 

FundingProfile 

1.25 

Li 

Change 

Baseline 

„T 

Sum  of  FieldedCount 

Row  Labels 

Column  Labels  V 

5 

6 

7 

8 

9 

10 

11 

12 

13 

14 

15 

16 

17 

18 

19 

20 

21 

22 

23 

24 

25 

26 

27 

28  Grand  Total 

M2A3  0IF 

216 

360 

288 

144 

144 

216 

72 

72 

72 

72 

72 

1728 

M2A3  ECP 

216 

360 

288 

144 

144 

144 

1296 

BMODIFV 

72 

72 

72 

72 

72 

72 

432 

M2A3ECP 

72 

72 

72 

72 

72 

72 

72 

144 

144 

144 

144 

144 

72 

1296 

BMODIFV 

72 

72 

72 

72 

72 

72 

72 

144 

144 

144 

144 

144 

72 

1296 

M3A3CAVOIF 

102 

170 

170 

68 

102 

68 

34 

34 

68 

816 

BMODCAV 

68 

34 

34 

34 

68 

238 

M3A3CAV  ECP 

102 

170 

170 

68 

34 

34 

578 

M3  A3  CAV  ECP 

68 

68 

BMODCAV 

68 

68 

M1A2SEP7USK 

140 

7D 

140 

140 

140 

70 

140 

140 

70 

70 

70 

70 

70 

70 

70 

70 

70 

70 

1680 

Ml  ECP 

140 

70 

140 

140 

140 

70 

140 

140 

70 

70 

1120 

AMOD 

70 

70 

70 

70 

70 

70 

70 

70 

560 
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Excursions  at  $1.5B 
Baseline  excursion  at  $1.5B 


MissionID 

(All) 

Ll 

FundingProfile 

1.5 

,T 

Change 

Baseline 

.1 

Sum  of  FieldedCount  Column  Labels  |_I_ 


Row  Labels  -T  5  6  7  8  9  10  11  12  13  14  15  16  17  18  19  20  21  22  23  24  25  26  27  28  GrandTotal 


M2A3CXF 

216 

360 

360 

144 

216 

216 

72 

72 

72 

1728 

M2A3ECP 

216 

36D 

360 

144 

144 

144 

1368 

BMODIFV 

72 

72 

72 

72 

72 

36D 

M2A3ECP 

72 

72 

72 

72 

72 

72 

72 

144 

144 

144 

144 

144 

72 

72 

1368 

BMODIFV 

72 

72 

72 

72 

72 

72 

72 

144 

144 

144 

144 

144 

72 

72 

1368 

M3A3  CAV  OIT 

102 

120 

170 

68 

68 

102 

68 

68 

816 

BMODCAV 

34 

34 

68 

68 

204 

M3A3CAV  ECP 

102 

170 

170 

68 

34 

68 

612 

M3A3CAVECP 

34 

34 

34 

34 

34 

34 

34 

34 

34 

34 

34 

34 

34 

68 

68  34 

612 

M1A2  SEP  TUSK 

140 

140 

140 

140 

140 

140 

140 

140 

70 

70 

70 

70 

70 

70 

70 

70 

1680 

Ml  ECP 

140 

140 

140 

140 

140 

140 

140 

140 

1120 

AMOD 

70 

70 

70 

70 

70 

70 

70 

70 

560 

Excursion  at  $1.5B,  without  RDTE  cost  considered 

The  GVC  alternative  is  only  funded  if  RDTE  costs  are  not  considered 


Sum  of  FieldedCount 

Column  Labels  Z_ 

Row  Labels 

5 

6 

7 

8 

9 

10 

11 

12 

13 

14 

15 

16 

17 

18 

19 

20 

21 

22 

23 

24 

25 

26 

27 

28  Grand  Total 

M2A30IF 

216 

360 

144 

144 

144 

144 

144 

144 

72 

72 

72 

72 

1728 

M2A3ECP 

216 

36D 

72 

72 

72 

72 

72 

72 

1008 

GCV 

72 

72 

72 

72 

72 

72 

72 

72 

72 

72 

720 

M2A3ECP 

72 

72 

72 

72 

72 

72 

72 

72 

144 

144 

72 

72 

1008 

GCV 

72 

72 

72 

72 

72 

72 

72 

72 

576 

BMODIFV 

144 

144 

72 

72 

432 

M3A3CAV  OIF 

102 

170 

68 

68 

68 

68 

68 

68 

34 

34 

34 

34 

816 

M3A3  CAV  ECP 

102 

170 

68 

68 

68 

68 

68 

68 

34 

34 

34 

34 

816 

M3A3CAV  ECP 

68 

68 

68 

68 

34 

34 

68 

68 

476 

M1A2  SEP  TUSK 

140 

140 

140 

140 

140 

140 

140 

140 

70 

70 

70 

70 

70 

70 

70 

70 

1680 

Ml  ECP 

140 

140 

140 

140 

140 

140 

140 

140 

1120 

AMOD 

70 

70 

70 

70 

70 

70 

70 

70 

560 

MissionID 

(All) 

Li 

FundingProfile 

1.5 

,T 

Change 

NoRDTECosts 

.T 

The  two  tables  for  Excursions  at  $1.5B  below  examine  how  changes  in  the  upper-bound 
of  the  number  of  M2A3  ECP  vehicles  that  may  be  purchased  in  one  period  affects  the 
solution.  The  first  table  shows  a  case  where  the  M2A3  ECP  fielding  is  limited  to  two 
BDEs  for  the  first  three  periods.  The  second  table  removes  the  BDE  limit  for  M2  A3 
ECP  fielding. 
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Excursions  at  $1.5B,  upper-bound  for  M2A3  ECP  vehicles  fielding  2BDE  Limit 


MissionID 

(All) 

Ll 

FundingProfile 

1.5 

.T 

Change 

2BDE  Limit 

JT 

Sum  of  FieldedCount  Column  Labels 


Row  Labels 

5 

6 

7 

8 

9 

10 

11 

12 

13 

14 

15 

16 

17 

18 

19 

20 

21 

22 

23 

24 

25 

26 

27  Grand  Total 

M2A3GU~ 

144 

144 

144 

144 

216 

216 

216 

216 

72 

144 

72 

1728 

M2A3  ECP 

144 

144 

144 

144 

144 

144 

144 

144 

1152 

BMOD  IFV 

72 

72 

72 

72 

72 

144 

72 

576 

M2A3ECP 

72 

72 

72 

72 

72 

72 

72 

144 

144 

144 

144 

72 

1152 

BMOD  IFV 

72 

72 

72 

72 

72 

72 

72 

144 

144 

144 

144 

72 

1152 

M3A3CAVOIF 

102 

102 

102 

68 

68 

102 

136 

136 

816 

BMOD CAV 

34 

68 

68 

68 

238 

M3A3  CAV  ECP 

102 

102 

102 

68 

34 

34 

68 

68 

578 

M3A3CAVECP 

68 

34 

34 

34 

34 

34 

34 

34 

34 

34 

34 

34 

34 

34 

68 

578 

M1A2  SEP  TUSK 

140 

140 

140 

140 

140 

70 

70 

70 

140 

70 

70 

70 

70 

70 

70 

70 

70 

70 

1680 

Ml  ECP 

140 

140 

140 

140 

140 

70 

70 

70 

140 

70 

1120 

AMOD 

70 

70 

70 

70 

70 

70 

70 

70 

560 

Excursions  at  $1.5B,  upper-bound  for  M2A3  ECP  vehicles  fielding  No  Limit 


MissionID 

(All) 

Ll 

FundingProfile 

1.5 

.T 

Change 

No  BDE  Limit 

,T 

Sum  of  FieldedCount 

Row  Labels 

Column  Labels  i Z_ 
5 

6 

7 

8 

9 

10 

11 

12 

13 

14 

15 

16 

17 

18 

19 

20 

21 

22 

23 

24 

25 

26 

27  Grand  Total 

M2A30F 

360 

360 

360 

360 

72 

72 

72 

72 

1728 

M2A3ECP 

360 

360 

360 

360 

1440 

BMOD  IFV 

72 

72 

72 

72 

288 

-  M2A3ECP 

72 

72 

72 

72 

72 

72 

72 

72 

72 

144 

144 

144 

144 

144 

72 

1440 

BMOD  IFV 

72 

72 

72 

72 

72 

72 

72 

72 

72 

144 

144 

144 

144 

144 

72 

1440 

M3A3CAVOIF 

170 

170 

170 

170 

68 

68 

816 

M3A3  CAV  ECP 

170 

170 

170 

170 

68 

68 

816 

M3A3CAVECP 

136 

136 

136 

136 

136 

34 

34 

34 

34 

816 

MIA 2 SEP  TUSK 

140 

140 

140 

140 

140 

140 

140 

140 

140 

140 

140 

140 

1680 

Ml  ECP 

140 

140 

140 

140 

70 

70 

70 

70 

70 

70 

70 

70 

1120 

AMOD 

70 

70 

70 

70 

70 

70 

70 

70 

560 

Baseline  excursion  at  S1.75B 


MissionID  (All) 

Ll 

FundingProfile  1.75 

.T 

Change  Baseline 

JT 

Sum  of  FieldedCount  Column  Labels 

Ll 

Row  Labels 

5 

6 

7 

8 

9 

10 

11 

12 

13 

14 

15 

16 

17 

18 

19 

20 

21 

22 

23 

24 

25 

26 

27 

28  Grand  Total 

M2A30F 

216 

360 

360 

144 

216 

216 

144 

72 

1728 

MM  3  ECP 

216 

360 

360 

144 

144 

144 

1368 

BMOD  IFV 

72 

72 

144 

72 

360 

M2A3bCP 

72 

72 

72 

72 

72 

72 

72 

72 

72 

72 

144 

144 

144 

144 

72 

1368 

BMOD  IFV 

72 

72 

72 

72 

72 

72 

72 

72 

72 

72 

144 

144 

144 

144 

72 

1368 

Ml  ECP 

70 

70 

70 

70 

70 

140 

140 

70 

700 

AMOD 

70 

70 

70 

70 

70 

140 

140 

70 

700 

M3A3CAV  Or 

102 

170 

170 

68 

102 

136 

34 

34 

816 

BMOD  CAV 

68 

68 

34 

34 

204 

M3A3CAV  ECP 

102 

170 

170 

68 

34 

68 

612 

M3A3CAV  t£P 

68 

68 

68 

68 

68 

68 

68 

68 

68 

612 

M1A2  SEP  TUSK 

140 

140 

140 

140 

140 

140 

140 

140 

70 

70 

70 

70 

70 

70 

70 

70 

1680 

Ml  ECP 

140 

140 

140 

140 

140 

140 

140 

140 

1120 

AMOD 

70 

70 

70 

70 

70 

70 

70 

70 

560 
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Excursions  at  $2.0B 

Baseline  excursion  for  the  $2.0B  funding  profile 


MissionID 

(All) 

FundingProfile 

2 

•T 

Change 

Baseline 

.T 

Sum  of  FieldedCount 

Column  Labels 

Ll 

Row  Labels 

5 

6 

7 

8 

9 

10 

11 

12 

13 

14 

15 

16 

17 

18 

19 

20 

Grand  Total 

-  M2A3  0IF 

216 

360 

144 

216 

216 

216 

144 

144 

72 

1728 

M2A3ECP 

216 

360 

72 

144 

144 

144 

1080 

GCV 

72 

72 

72 

72 

144 

144 

72 

648 

-  M2A3  ECP 

72 

144 

144 

144 

144 

144 

144 

144 

1080 

GCV 

72 

144 

144 

144 

144 

144 

144 

144 

1080 

-  M3A3CAVOIF 

102 

170 

170 

68 

68 

68 

646 

M3A3CAV  ECP 

102 

170 

170 

68 

68 

68 

646 

M1A2  SEP  TUSK 

140 

140 

140 

140 

140 

70 

70 

70 

70 

70 

70 

1120 

Ml  ECP 

140 

140 

140 

140 

140 

70 

70 

70 

70 

70 

70 

1120 
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Excursions  at  $2.0B,  increase  10%  for  given  alternative 

The  three  tables  below  illustrate  how  a  10%  increase  in  value  for  a  given  alternative  in 
one  role  may  affect  the  alternative  in  another  role.  In  the  first  table,  AMOD  is  increased 
by  10%,  followed  by  M1ECP  in  the  second  table  and  M2AECP  in  the  third  table. 


MissionID 

(All) 

FundingProfile 

2 

,T 

Change 

+10%AMOD 

Sum  of  FieldedCount  Column  Labels 


Row  Labels 

5 

6 

7 

8 

9 

10 

11 

12 

13 

14 

15 

16 

17 

18 

19 

20 

Grand  Total 

-  M2A3  OIF 

216 

360 

144 

216 

216 

216 

144 

144 

72 

1728 

M2A3ECP 

216 

360 

72 

144 

144 

144 

1080 

GCV 

72 

72 

72 

72 

144 

144 

72 

648 

-  M2A3  ECP 

72 

144 

144 

144 

144 

144 

144 

144 

1080 

GCV 

72 

144 

144 

144 

144 

144 

144 

144 

1080 

-  M3A3CAVOIF 

102 

170 

170 

68 

68 

68 

646 

M3A3CAV  ECP 

102 

170 

170 

68 

68 

68 

646 

M1A2  SEP  TUSK 

140 

140 

140 

140 

140 

70 

70 

70 

70 

70 

70 

1170 

Ml  ECP 

140 

140 

140 

140 

140 

70 

70 

70 

70 

70 

70 

1120 

MissionID 

(All) 

FundingProfile 

2 

•T 

Change 

+10%M1ECP 

Sum  of  FieldedCount 

Column  Labels 

[z_ 

Row  Labels 

5 

6 

7 

8 

9 

10 

11 

12 

13 

14 

15 

16 

17 

18 

19 

20 

Grand  Total 

-  M2A3  0IF 

216 

360 

144 

216 

216 

216 

144 

144 

72 

1728 

M2A3  ECP 

216 

360 

72 

144 

144 

144 

1080 

GCV 

72 

72 

72 

72 

144 

144 

72 

648 

-  M2A3  ECP 

72 

144 

144 

144 

144 

144 

144 

144 

1080 

GCV 

72 

144 

144 

144 

144 

144 

144 

144 

1080 

-  M3A3CAVOIF 

102 

170 

170 

68 

68 

68 

646 

M3A3CAV  ECP 

102 

170 

170 

68 

68 

68 

646 

M1A2  SEP  TUSK 

140 

140 

140 

140 

140 

70 

70 

70 

70 

70 

70 

1120 

Ml  ECP 

140 

140 

140 

140 

140 

70 

70 

70 

70 

70 

70 

1120 

MissionID 

(All)  Ll 

FundingProfile 

2  [£ 

Change 

+10%M2A2ECP  £L 

Sum  of  FieldedCount 

Column  Labels 

Li. 

Row  Labels 

5 

6 

7 

8 

9 

10 

11 

12 

13 

14 

15 

16 

17 

18 

19 

20 

Grand  Total 

-  M2A3  OIF 

216 

360 

288 

216 

216 

216 

144 

72 

1728 

M2A3ECP 

216 

360 

216 

144 

144 

144 

1224 

GCV 

72 

72 

72 

72 

144 

72 

504 

-  M2A3  ECP 

72 

144 

144 

144 

144 

144 

144 

144 

144 

1224 

GCV 

72 

144 

144 

144 

144 

144 

144 

144 

144 

1224 

-  M3A3  CAVOIF 

102 

170 

170 

68 

68 

68 

646 

M3A3CAV  ECP 

102 

170 

170 

68 

68 

68 

646 

M1A2  SEP  TUSK 

140 

70 

140 

140 

140 

70 

70 

70 

70 

70 

70 

70 

1170 

ML  ECP 

140 

70 

140 

140 

140 

70 

70 

70 

70 

70 

70 

70 

1120 
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Excursions  at  $2.0B,  increase  10%  for  all  alternatives  in  MBT  role 

In  this  excursion,  we  increase  all  alternatives  contained  within  the  MBT  role  by  10%. 
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Excursions  at  $2. OB,  changes  in  upper-bound  of  M2A3  ECP  vehicles 

The  two  tables  for  Excursions  at  $2. OB  below  examine  how  changes  in  the  upper-bound 
of  the  number  of  M2A3  ECP  vehicles  that  may  be  purchased  in  one  period  affects  the 
solution.  The  first  table  shows  a  case  where  the  M2A3  ECP  fielding  is  limited  to  two 
BDEs  for  the  first  three  periods.  The  second  table  removes  the  BDE  limit  for  M2  A3 
ECP  fielding. 
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Baseline  excursions  at  $2.6B 
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APPENDIX  F.  PROGRAM  EXECUTIVE  OFFICE  (PEO) 
ANALYSIS  TEAM  RESPONSE  TO  FINAL  DRAFT 


Below  are  the  relevant  comments  received  from  the  sponsor  in  support  of  CP  AT. 

1.  As  part  of  our  CP  AT  FY12  data  refresh,  the  PEO  analysis  team  is 
re-evaluating  the  measures  to  address  the  concern  presented  on  page  2 
regarding  ineffective  value  measures. 

2.  CP  AT  limitations  taken  from  previous  PEO  documents  stated  that  the 
team  should  have  conducted  two  additional  panels.  The  later  response 
from  the  PEO  analysis  team  was  that  one  panel  was  enough  as  long  as  it 
was  the  right  group. 

3.  The  PEO  analysis  team  initially  interpreted  “trade-offs”  mentioned  in  the 
document  to  mean  the  force  structure  input  by  TRADOC.  Further 
discussion  clarified  that  trade-offs  in  this  document  refer  to  the  swing 
weights,  specifically  how  the  20  performance  models  are  weighted  against 
each  other  by  the  optimization  model. 

4.  The  PEO  analysis  team  is  making  adjustments  to  the  value  hierarchy  that 
will  result  in  a  revaluation  and  reweighting  of  attributes. 

5.  The  PEO  analysis  team  identified  an  error  in  our  initial  value  function 
sensitivity  analysis.  The  corrected  analysis  has  been  incorporated  in  this 
document. 

6.  The  PEO  is  working  with  AMSAA  as  part  of  the  FY12  data  refresh  to 
refine  specific  value  functions. 

7.  Value  function  scale  much  larger  than  existing  data  because  the  PEO 
wants  flexibility  to  measure  future  systems  with  the  same  model.  We 
propose  that  it  is  better  to  have  “tighter”  scale  when  possible.  If  the  scale 
needs  to  be  expanded  in  the  future,  then  do  so.  Then  to  compare  with  past 
results,  just  run  old  alternatives  in  the  new  model. 

8.  NPS  and  the  PEO  analysis  team  discussed  the  optimization  fonnulation  in 
a  meeting  subsequent  to  the  review  of  the  final  draft  of  this  document. 
Most  issues  or  confusion  over  how  the  fonnulation  is  implemented  were 
resolved  during  this  meeting.  However,  several  are  left  unresolved.  For 
example,  we  disagree  with  how  the  elastic  constraints  are  used  but  this 
should  not  impact  the  execution  of  the  model  in  practice  as  they  are  not 
used  by  the  PEO  analysis  team. 

9.  Many  of  the  most  significant  findings  of  this  V&V  are  discussed  in  the 
implementation  review.  Most  of  the  recommendations  are  accepted  by  the 
PEO  analysis  team  and  they  plan  to  implement  these  recommendations  in 
later  versions  of  CP  AT. 
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